GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
r0b0tlab
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
r0b0tlab
Model Tree
XYZAILab/XYZ-Aquila-mini
Input Modalities
Output Modalities
Supported Functionality
| Item | Value |
|---|---|
| Source | XYZAILab/XYZ-Aquila-mini @ 4d3b0c45… |
| Architecture | Qwen3_5MoeForConditionalGeneration / qwen3_5_moe |
| Quant recipe | Mixed: NVFP4 W4A4 on MoE MLP / shared experts / eligible linears; FP8 on attention projections; FP8 KV |
| Quantized layers | 161 NVFP4 + 130 FP8 |
| Artifact size | ~21 GiB (3 safetensors shards) |
| BF16 reference size | ~70.2 GiB parameter bytes (CPU sequential load audit) |
| MTP | Disabled in derived config (no borrowed MTP tensors) |
| Calibration | 512 windowed rows / 253,409 tokens (Aquila-SFT derived; biased fulltext-first rejected) |
Use a vLLM 0.25 image with native SM121 NVFP4 kernels (FlashInfer CUTLASS NVFP4 + FlashInfer B12X MoE). Companion runtime:
ghcr.io/r0b0tlab/xyz-aquila-mini-nvfp4-vllm:v0.25.0-sm121-702f4814bash
vllm serve /path/to/XYZ-Aquila-mini-NVFP4 \--served-model-name xyz-aquila-nvfp4 \--quantization modelopt_mixed \--kv-cache-dtype fp8 \--attention-backend flashinfer \--moe-backend flashinfer_b12x \--linear-backend flashinfer_cutlass \--language-model-only \--enable-auto-tool-choice \--tool-call-parser qwen3_xml \--reasoning-parser qwen3 \--gpu-memory-utilization 0.85 \--max-model-len 32768 \--max-num-seqs 16 \--max-num-batched-tokens 16384 \--trust-remote-code
FlashInferCutlassNvFp4LinearKernel for NVFP4 GEMMFLASHINFER_B12X MoE backend (not Marlin / emulation)arch=sm121Do not treat export metadata alone as native kernel proof.
All scores are from this checkpoint + the companion SM121 runtime unless noted.
bfcl-eval==2025.12.17)| Suite | Result |
|---|---|
| multi_turn_base (200) | 128/200 = 64.00% |
| AST multiple (200) | 70/200 = 35.00% |
| AST parallel (200) | 93/200 = 46.50% |
| AST parallel_multiple (200) | 37/200 = 18.50% |
| AST micro-avg (600) | 200/600 = 33.33% |
Protocol notes: FC handler, temperature 0.001, threads=1; AST project isolated from multi-turn; 0 inference-error rows on final artifacts. Leaderboard “Overall Acc” columns that include unscored categories are not the headline metrics above.
identity / zh_arithmetic / structured JSON / tool_call / needle — PASS (thinking models may place answers after reasoning; checks accept final content).
vllm bench serve random 1024in/256out, ignore_eos, temp 0, 3 reps drop-first; util 0.85, max_num_seqs 16.
| Conc. | Output tok/s | Total tok/s | mean TTFT ms | mean TPOT ms | Failed |
|---|---|---|---|---|---|
| 1 | 64.6 | 325.4 | 694 | 12.8 | 0 |
| 2 | 82.4 | 415.3 | 2010 | 16.3 | 0 |
| 4 | 164.5 | 829.1 |
Full runtime docs: https://github.com/r0b0tlab/xyz-aquila-mini-nvfp4-sm121-vllm
text
config.jsongeneration_config.jsonhf_quant_config.jsonchat_template.jinjatokenizer.jsontokenizer_config.jsonmodel.safetensors.index.jsonmodel-00001-of-00003.safetensorsmodel-00002-of-00003.safetensorsmodel-00003-of-00003.safetensors
max_tokens for short-form tasks.Weights are a derivative of XYZAILab/XYZ-Aquila-mini. Respect the upstream license and terms. Quantization tooling: NVIDIA ModelOpt. Packaging/runtime docs: Apache-2.0 where original to this release.
4d3b0c45663b8b40f9e6e57b64ff815c09c49904ghcr.io/r0b0tlab/vllm-v0250-cu130-sm121@sha256:2d144fafe3f330fa17fa1facf4f589eee49b75bdf539ac69d1fe002b5b5bb0a5702f4814fe54fabff350d43cb753ae3e47c0c276Upstream XYZAILab for Aquila-mini; NVIDIA ModelOpt / vLLM / FlashInfer communities; r0b0tlab SM121 runtime packaging.
| 641 |
| 21.9 |
| 0 |
| 8 | 230.5 | 1162.1 | 1009 | 30.9 | 0 |
| 16 | 305.7 | 1540.9 | 1854 | 45.3 | 0 |