GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
vroomfondel
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
vroomfondel
Model Tree
deepreinforce-ai/Ornith-1.0-9B
Input Modalities
Output Modalities
Supported Functionality
Serving this checkpoint correctly on SGLang needs two fixes, both applied at launch by the serving launch script:
in_proj_qkv NVFP4 scale loads into only one slot, corrupting the global
alpha, and the model emits word-salad. This is the critical one for this build.qwen3_5.py,
sgl-project/sglang#31220:
enable quantized attention for modelopt_fp4 + load the baked FP8 KV-cache scales.Serve on Blackwell (e.g. DGX Spark GB10 / sm121) with:
markdown
--quantization modelopt_fp4 \--attention-backend flashinfer \--fp4-gemm-backend flashinfer_cutlass \--kv-cache-dtype fp8_e4m3 \--reasoning-parser qwen3 \--tool-call-parser qwen3_coder
See the Notes section below for the exact patch details.
nvfp4 kv_cache: fp8512 samples from each of pg19, cnn_dailymail0.45.0Before/after sample generation was skipped for this run (SKIP_GENERATE=1).
This checkpoint quantizes attention as well as the FFN (uniform W4A4), unlike NVIDIA's own Qwen3.5 NVFP4 exports which keep attention in BF16. SGLang's qwen3_5.py hardcodes attention to unquantized for any modelopt_fp4 checkpoint, so it must be patched at launch time. This is applied via quantizer/sglang_launch_sm121.sh (a Python heredoc that flips the forced-None override so ModelOptFp4Config.is_layer_excluded() decides per-prefix instead). Upstreamed in sgl-project/sglang PR #31220. The patch is a no-op on NVIDIA/attention-BF16 checkpoints.
This build quantizes the Gated-DeltaNet linear-attention path, whose fused in_proj_qkv loads its per-tensor NVFP4 scale (weight_scale_2 / input_scale) via a tuple shard id. SGLang before PR #29151 (commit cfc3d0555e, merged to main, NOT in 0.5.15) hardcodes shard_id=0 for that scalar in MergedColumnParallelLinear / QKVParallelLinear.weight_loader_v2, leaving the other logical slots uninitialized; the ModelOpt NVFP4 path then .max()es over them so a garbage slot becomes the runtime global alpha, producing word-salad. Serve on a SGLang that includes PR #29151, or apply it at launch (this build's launch script does). Confirmed against a BF16 baseline and a linear-attn-excluded NVFP4 build (both coherent): the failure is this scale-loading path, not the recipe.
This is the uniform W4A4 variant: the dense FFN, the full attention, AND the Gated-DeltaNet linear-attention path are all NVFP4 (FP8 KV cache). It has cleared a coherence smoke-serve (capital-of-Australia, arithmetic, code generation all correct) and GSM8K (94.0%, see Benchmarks), with the two serving fixes above applied. Unlike some other NVFP4 Qwen3.5 builds, it does quantize the linear-attention path.
Calibration is text-only, so the vision tower is excluded from quantization and stays BF16 (avoiding the amax=0 degenerate-quant failure mode). The export ships the processor configs (preprocessor_config.json etc.) needed to serve this as a proper multimodal checkpoint.
| Task | Metric | Value | Setup | Hardware | Date | Notes |
|---|---|---|---|---|---|---|
| GSM8K | accuracy (strict == flexible) | 94.0% | chat + reasoning (qwen3 think), 4-shot, temperature 0.6, max_tokens 4500, N=400 | DGX Spark GB10 / sm121; SGLang 0.5.15 + NVFP4 scalar-scale fix (PR #29151) + qwen3_5 attention/KV patches (PR #31220) | 2026-07-15 | 376/400 correct; strict == flexible; 11/400 (2.75%) empty (generation-length cap). Uniform W4A4 (dense FFN + full attention + Gated-DeltaNet linear-attn NVFP4), FP8 KV cache. |