GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
inference-optimization
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
inference-optimization
Model Tree
moonshotai/Kimi-K2.5
Input Modalities
Output Modalities
Supported Functionality
The following parameters were reduced from the original model:
| Parameter | Original | Tiny |
|---|---|---|
text_config.num_hidden_layers | 61 | 3 (1 dense + 2 MoE) |
text_config.hidden_size | 7168 | 1024 |
text_config.intermediate_size | 18432 | 2048 |
text_config.moe_intermediate_size | 2048 | 512 |
text_config.num_attention_heads | 64 | 8 |
text_config.num_key_value_heads | 64 | 8 |
text_config.q_lora_rank | 1536 | 256 |
text_config.kv_lora_rank | 512 | 128 |
text_config.qk_nope_head_dim | 128 | 64 |
text_config.qk_rope_head_dim | 64 | 32 |
text_config.qk_head_dim | 192 | 96 |
text_config.v_head_dim | 128 | 64 |
text_config.n_routed_experts | 384 | 8 |
text_config.num_experts_per_tok | 8 | 4 |
vision_config.num_hidden_layers | 27 | 2 |
vision_config.vt_num_hidden_layers | 27 | 2 |
text_config.quantization_config | int4 pack-quantized | removed (unquantized) |
Architecture-preserving choices: first_k_dense_replace=1 (1 dense layer then MoE), MLA attention (multi-head latent attention with LoRA-style KV compression), sigmoid scoring + noaux_tc routing, shared experts, vision tower + patchmerger projector.
Single-file checkpoint (model.safetensors) with 126 tensors. Weight naming matches the original model structure:
language_model.model.* — text transformer (DeepSeek V3 with MLA + MoE)vision_tower.* — vision encoder (MoonViT3d)mm_projector.* — vision-to-text projection (patchmerger)The only structural difference from the original: expert weights are stored as .weight (unquantized float) instead of .weight_packed / .weight_scale / .weight_shape (4-bit compressed).
python
from transformers import AutoModelForCausalLM, AutoTokenizermodel = AutoModelForCausalLM.from_pretrained("inference-optimization/Kimi-K2.5-0.5B",trust_remote_code=True,attn_implementation="eager",device_map="auto",)tokenizer = AutoTokenizer.from_pretrained("inference-optimization/Kimi-K2.5-0.5B",trust_remote_code=True,)input_ids = tokenizer("According to all known laws", return_tensors="pt").input_ids.to(model.device)output = model.generate(input_ids, max_new_tokens=20, use_cache=False)print(tokenizer.decode(output[0]))
Note: attn_implementation="eager" is required for compatibility with transformers v5. use_cache=False is recommended until KV cache compatibility with transformers v5 is resolved.
This model was created using the llm-compressor create-tiny-model Claude skill.
Steps:
AutoConfig.from_pretrainedDynamicCache.from_legacy_cache, is_torch_fx_available, fixed MoE training-mode forward, tie_weights signature)markdown
Success: 1.0005239248275757 <= 10.0According to all known laws of aviation, there is no way a bee should be able to fly.
input_ids only (no pixel_values). The vision tower is present but not invoked for text-only tasks.trust_remote_code=True due to custom modeling code for KimiK25 and DeepSeek V3 architectures.moonshotai/Kimi-K2.5 model code targets transformers v4).