GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
fraserprice
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
fraserprice
Model Tree
deepreinforce-ai/Ornith-1.0-397B
Input Modalities
Output Modalities
Supported Functionality
bash
hf download fraserprice/Ornith-1.0-397B-W4A16-AutoRound-DFlash --local-dir Ornith-1.0-397Bcd Ornith-1.0-397B./serve-rtx-pro.sh
OpenAI-compatible server on port 8000: TP4, 262144 context, reasoning parser qwen3, tool parser
qwen3_xml. Runs on voipmonitor/vllm:eldritch-enlightenment-v2226f26-b12x15cd38c-cu132-20260629
(vLLM 0.11.2.dev + native support for the DFlash sliding-window drafter). First run pulls the image
and downloads the draft weights. Override via PORT, TP, MODEL_DIR, DRAFT_DIR, IMAGE.
--attention-backend FLASHINFER: split-KV decode keeps the multi-token verify step at ~20 ms
per step regardless of context length.num_speculative_tokens=12: ~40% faster decode than k=8 at concurrency 1, equal at 2.
k=16 (the draft's native block) is slower.--async-scheduling off; both alternatives lose speed and/or
acceptance on this hybrid-GDN architecture.NCCL_P2P_LEVEL=SYS + VLLM_MARLIN_USE_ATOMIC_ADD=1: +13% decode, +40% prefill on 4× PCIe GPUs.4× RTX Pro 6000 Blackwell, TP4, streaming OpenAI requests, real coding prompts generated to natural EOS (≤2048 tokens), 3 repeats. Vanilla = same image and quant, default vLLM serving per the base model card, no speculative decoding. Accept = mean accepted tokens per verify step.
| prompt | conc | decode tok/s | vanilla tok/s | uplift | accept | TTFT p50 | ITL p50 |
|---|---|---|---|---|---|---|---|
| 1,000 | 1 | 197.7 | 121.2 | 1.63× | 4.20 | 152 ms | 21.0 ms |
| 10,000 | 1 | 139.5 | 119.9 | 1.16× |
Prefill throughput 6.7–8.7k tok/s.
| 3.71 |
| 1.28 s |
| 21.1 ms |
| 100,000 | 1 | 148.5 | 107.9 | 1.38× | 3.41 | 14.2 s | 22.6 ms |
| 1,000 | 3 | 149.1 | 80.1 | 1.86× | 4.20 | 392 ms | 29.2 ms |
| 10,000 | 3 | 104.7 | 75.6 | 1.38× | 3.45 | 2.86 s | 29.7 ms |
| 100,000 | 3 | 69.8 | 47.1 | 1.48× | 3.61 | 28.1 s | 33.6 ms |