GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
joerunde
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
joerunde
Model Tree
Input Modalities
Output Modalities
Supported Functionality
spyre-inference validates its Spyre lowering per model architecture. The real
gemma-4 MoE checkpoints are 26B+ parameters and dominate CI wall-clock, yet no small
public gemma-4 MoE checkpoint satisfies Spyre's head-size-multiple-of-64 rule (every
public tiny-random one uses head_dim 16/32). This model fills that gap: it carries
the exact gemma-4 architecture at a few-hundred-MB size, so the architecture can
be exercised on every PR while the full product models run in an opt-in suite.
Gemma4ForConditionalGeneration / model_type: gemma4, text + vision towerlayer_types mix (with a full_attention layer retained)Shrunk (size only): hidden size, FFN / MoE intermediate sizes, text layer count (→ 6, keeping one full-attention layer), and vision layer count.
Reproducibly, by tests/data/generate_micro_gemma4_moe.py in spyre-inference:
it loads the real config, overrides only the size fields, applies controlled init
plus a short language-modeling conditioning pass (so the output distribution is
non-flat and numerically well-behaved), and verifies a finite fp16 forward.
Loaded and run compiled on a Spyre card via the text-backbone path
(hf_overrides={"architectures": ["Gemma4ForCausalLM"]}), with no CPU fallback —
the shrunk config lowers through the same SpyreGemma4ForCausalLM per-block compile
as the full model.