GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
Ar4ikov
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Ar4ikov
Model Tree
deepreinforce-ai/Ornith-1.0-9B
Input Modalities
Output Modalities
Supported Functionality
This is a hybrid model (mixed linear_attention / full_attention layers). The recipe
follows the proven approach for this architecture: quantize the standard projections, keep the
sensitive state-space gates and non-LLM parts in BF16.
Quantized to INT4 (200 Linear modules):
q_proj / k_proj / v_proj / o_proj (8 layers)gate_proj / up_proj / down_proj (32 layers)out_proj, in_proj_qkv, in_proj_z (24 layers)Kept in BF16 (ignore, 159 Linear modules):
re:model\.visual\..* (110)in_proj_a / in_proj_b (48) — sensitive to the recurrencelm_head, and any mtp headbash
vllm serve Ar4ikov/Ornith-1.0-9B-AWQ-W4A16-ASYM --trust-remote-code
Expected GPU memory ≈ checkpoint size + KV cache + activations.
The model emits <think>...</think> reasoning blocks before the final answer.
markdown
temperature = 0.6, top_p = 0.95, top_k = 20
The recipe.yaml written by llm-compressor is included in this repo. config.json carries the
quantization_config (compressed-tensors, format pack-quantized) describing the exact
scheme, group size and ignore list.
lm_head and the vision tower hurts quality for little gain, so they stay BF16.
Together with the large embedding table and the BF16 SSM gates, this is why a 9B hybrid
compresses to ~8 GB rather than the ~4.5 GB of a pure-transformer 7B.transformers transparently decompresses the weights
back to BF16, so VRAM there looks like the original model. The real INT4 memory saving is
realized by vLLM's native 4-bit kernels.