GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
protoLabsAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
protoLabsAI
Model Tree
Base
deepreinforce-ai/Ornith-1.0-35B
Quantized
this model
Input Modalities
Text
Image
Video
Output Modalities
Text
Supported Functionality
Dedicated Endpoints
The upstream FP8 release was reported broken. This is a clean, serve-validated FP8 built to the Qwen official block-wise format (served natively on vLLM's fused-MoE Triton path), with the entire linear-attention / SSM path and the MoE router kept in bf16 — FP8-quantizing those corrupts the hybrid SSM and is the most likely failure mode for a naive FP8 of this architecture.
[128, 128] FP8 E4M3, dynamic per-token activations (quant_method: fp8, activation_scheme: dynamic).modules_to_not_convert): lm_head, embed_tokens, MoE router gates (mlp.gate, shared_expert_gate), all linear_attn.* (SSM: in_proj_*, out_proj, conv1d, A_log, dt_bias), all norms, and the vision tower.Smoke: coherent across coding / reasoning / long-form / multilingual / tool-calling — no gibberish, no reasoning leak.
| Metric | bf16 source | This FP8 |
|---|---|---|
| custom coding (one-shot) | 1.00 (10/10) | 0.975 (9/10) |
| function-call | 93% (50/54) | 89% (48/54) |
| decode tok/s (1× RTX PRO 6000) | 208 | 207 |
Deltas are within single-trial run-to-run variance (temp 0.6–0.7); throughput is identical to the source.
bash
vllm serve protoLabsAI/Ornith-1.0-35B-FP8 \--served-model-name ornith-35b \--max-model-len 262144 \--reasoning-parser qwen3 \--enable-auto-tool-choice --tool-call-parser qwen3_xml \--gpu-memory-utilization 0.90 \--trust-remote-code
Context: full 256K (262144). Vision: the base is multimodal (Qwen-VL-style image + video tokens) and the vision tower is preserved in bf16 — the recipe above keeps it enabled. For text-only serving (smaller footprint), add --language-model-only. Verified serving with vision on at 256K on RTX PRO 6000 (Blackwell, sm120).
Ornith is a reasoning model: the assistant turn opens with a <think>…</think> block surfaced as reasoning_content; tool calls are emitted as standard tool_calls. Recommended sampling: temperature=0.6, top_p=0.95, top_k=20.
MIT, inheriting Ornith-1.0. All credit for the base model to the DeepReinforce team (blog); this repo only adds the FP8 quantization.
bibtex
@misc{ornith-35b, title={{Ornith-1.0-35B}: Agentic Coding, Open to All},url={https://deep-reinforce.com/ornith_1_0.html}, author={{DeepReinforce Team}}, year={2026}}
Ornith-1.0-9B-NVFP4 — calibrated
W4A4 for vLLM, MTP sidecar in-box; 10.4 GB, gate-verified parity, ~1.5x bf16+MTP.Ornith-1.0-9B-MTP-GGUF — llama.cpp
builds incl. the NVFP4+MTP rung (306 tok/s on Blackwell).Ornith-1.0-9B-MTP — the MTP draft head.Benchmark rows: protoLabsAI/lab-benchmarks ·
protolabs.studio/lab. Different quant? Community discussion — ~48h.