GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Capicua25x
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Capicua25x
Model Tree
ornith-ai/Ornith-1.0-9B
Input Modalities
Output Modalities
Supported Functionality
2× Radeon AI PRO R9700 (gfx1201), tensor-parallel 2, vLLM 0.19.1 (image below).
qwen3_xml) and vision confirmed working.| concurrent | short prompt | ~6k prompt |
|---|---|---|
| 1 | 84.6 / 85 | 103.8 / 104 |
| 16 | 51.8 / 803 | 50.1 / 767 |
| 32 | 40.8 / 1260 | 33.0 / 1010 |
| 64 | 30.7 / 1902 | 21.1 / 1268 |
| 96 | 22.5 / 2075 | 15.0 / 1293 |
| 128 | 20.2 / 1893 | 12.6 / 1284 |
Usable concurrency ceiling (per-user ≥ 20 tok/s): ~128 at short context, ~64 at 6k context. Single-stream decode is bound by the dense 9B's active-parameter count; a single GPU (TP1) is also supported.
Uses the prebuilt RDNA4 vLLM image
capicua25x/vllm-rocm-rdna4 (tag 0.19.1):
bash
docker run --rm --network=host \ --device=/dev/kfd --device=/dev/dri \ --group-add=video --group-add=render --ipc=host --ulimit memlock=-1 \ capicua25x/vllm-rocm-rdna4:0.19.1 \ --model Capicua25x/Ornith-1.0-9B-MXFP4-Vision-MTP \ --served-model-name ornith --trust-remote-code \ --tensor-parallel-size 2 \ --gpu-memory-utilization 0.90 \ --max-model-len 16384 \ --attention-backend TRITON_ATTN \ --enable-prefix-caching \ --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3 \ --speculative-config '{"method":"mtp","num_speculative_tokens":3}'--attention-backend TRITON_ATTN — required on gfx1201.--speculative-config '{"method":"mtp","num_speculative_tokens":3}' — enables the grafted MTP head.
n=3 maximizes throughput; n=1–2 maximize per-token acceptance. Tune per workload.--tool-call-parser qwen3_xml --reasoning-parser qwen3 — Qwen3.5-family tool-calling + reasoning split.--trust-remote-code — the qwen3_5 vision architecture.--tensor-parallel-size 1 and pass one render node
(e.g. --device=/dev/dri/renderD128).compressed-tensors (4-bit float, group 32, symmetric;
lm_head / embed_tokens / norms / vision tower left BF16).mtp.* head tensors from protoLabsAI/Ornith-1.0-9B-MTP into a new
model-mtp.safetensors shard and patch model.safetensors.index.json.mtp.fc, mtp.layers.0.self_attn.*,
mtp.layers.0.mlp.*) to quantization_config.ignore, so vLLM's compressed-tensors loader keeps the
BF16 head as-is instead of expecting MXFP4 weight-scales. This is the one mixed-precision gotcha.Step 1 (the MXFP4 quantize) is scripted in quantize_mxfp4.py (weight-only MXFP4A16, group 32, data-free — deterministic/byte-reproducible). Steps 2–3 are scripted in recipe_graft_mxfp4.py (run against an MXFP4
compressed-tensors trunk + the protoLabs head). The head's distillation recipe lives upstream at
protoLabsAI/Ornith-1.0-9B-MTP.
Ornith-1.0-9B, the base model (MIT).Ornith-1.0-9B-MTP, the KL-distilled MTP draft head and its recipe (MIT).Qwen/Qwen3.5-9B's mtp.* tensors).tcclaviger) — the RDNA4 vLLM base image that made gfx1201 serving possible. The image this model runs on is a forward-port of his work — without it, none of this runs.MIT. This is a derivative of Ornith-1.0-9B (MIT); merging the MTP head (MIT) produces a derivative whose MIT terms carry.
capicua25x/vllm-rocm-rdna4