GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
inference-optimization
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
inference-optimization
Model Tree
Qwen/Qwen3.6-35B-A3B
Input Modalities
Output Modalities
Supported Functionality
Qwen3_5MoeForConditionalGeneration)This is a multimodal (vision-language) mixture-of-experts model with a hybrid linear-attention / full-attention text backbone. The tiny model preserves the full architecture of the original: hybrid attention pattern (linear + full), 256 routed experts with top-8 routing, a shared expert, and the vision tower.
Only depth was reduced; all widths (hidden size, expert count, MoE intermediate size, attention head dims, vocab) match the original to keep the architecture faithful.
| Parameter | Original | Tiny |
|---|---|---|
text_config.num_hidden_layers | 40 | 8 |
text_config.layer_types | 30 linear + 10 full | 6 linear + 2 full |
vision_config.depth | 27 | 2 |
text_config.num_experts | 256 | 256 (unchanged) |
text_config.num_experts_per_tok | 8 | 8 (unchanged) |
text_config.moe_intermediate_size | 512 | 512 (unchanged) |
text_config.hidden_size | 2048 | 2048 (unchanged) |
text_config.vocab_size | 248320 | 248320 (unchanged) |
The hybrid attention pattern (full_attention_interval=4) is preserved: text
layers are [linear, linear, linear, full, linear, linear, linear, full],
giving 6 linear-attention and 2 full-attention layers.
Sharded safetensors checkpoint with model.safetensors.index.json, matching the
original repository layout. Routed experts are stored in the original fused
format (one 3D tensor per layer):
...mlp.experts.gate_up_proj → [256, 1024, 2048]...mlp.experts.down_proj → [256, 2048, 512]All non-MTP tensor name patterns match the original checkpoint exactly. The MTP (multi-token-prediction) layers from the original are intentionally omitted.
python
import torchfrom transformers import Qwen3_5MoeForConditionalGeneration, AutoTokenizermodel = Qwen3_5MoeForConditionalGeneration.from_pretrained("inference-optimization/Qwen3.6-8B-A1.6B", dtype=torch.bfloat16, device_map="auto")tokenizer = AutoTokenizer.from_pretrained("inference-optimization/Qwen3.6-8B-A1.6B")input_ids = tokenizer("According to all known laws", return_tensors="pt").input_ids.to(model.device)output = model.generate(input_ids, max_new_tokens=20)print(tokenizer.decode(output[0]))
This model was created using the llm-compressor create-tiny-model claude skill.
Qwen3_5MoeForConditionalGeneration (no missing/unexpected keys).