GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
MIRALABS
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
MIRALABS
Model Tree
ornith-ai/Ornith-1.5-35B-A3B
Input Modalities
Output Modalities
Supported Functionality
Starting from the author's asymmetric checkpoint, each expert projection was:
W = (q − zp)·s (original bf16 scale).s_new = amax(W, per 32-group) / 7.5 (stored bf16),
q = clamp(round(W / s_new), −8, 7), repacked 8 codes per int32.symmetric: true, zp_dtype: null.vLLM reconstructs each weight as (q − 8)·s_new. The cost is one extra
re-quantization step (~amax/15 per group) on top of the original AWQ
quantization. Everything non-quantized (BF16 vision tower, linear/full
attention, shared experts, norms, lm_head, embeddings, biases, A_log/
dt_bias/conv1d, and the full BF16 MTP head) is byte-identical to the
source. The conversion script is ornith_asym_to_sym.py (deterministic,
resumable, validates that no zero-points and no 3-D packed tensors remain).
bash
vllm serve /path/to/Ornith-1.5-35B-A3B-W4A16-SYM \--tensor-parallel-size 2 \--max-model-len 32768 \--gpu-memory-utilization 0.90 \--host 127.0.0.1 --port 9001 \--limit-mm-per-prompt '{"image":1,"video":1}' \--mm-processor-kwargs '{"max_pixels":401408}'
On two RTX 3090 (Ampere) it loads with
markdown
Using Marlin backend for WNA16 MoE (group_size=32, num_bits=4)
--kv-cache-dtype auto on Ampere (no fp8). The reasoning/tool parsers are
model-built-in for this first-serve config.
Verified against the release coherence gate on vLLM: multi-step arithmetic (correct), syllogistic reasoning (correct), memoized Python (correct), and a basic color image (correct dominant color). As with any 4-bit build, greedy output diverges from BF16; this is a 4-bit model.
ornith-ai/Ornith-1.5-35B-A3B (MIT), a fine-tune of the Qwen3.6/3.5 A3B
MoE family.ulkaa/Ornith-1.5-35B-A3B-AWQ-INT4 (MIT).MIT, inherited from ornith-ai/Ornith-1.5-35B-A3B.