GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
EmbeddedLLM
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
EmbeddedLLM
Model Tree
thinkingmachines/Inkling-Small
Input Modalities
Output Modalities
Supported Functionality
0.12.post1+rocm72.torch2.11)
This checkpoint was built from thinkingmachines/Inkling-Small revision
b2d4f225a02032c5d154bff748ab5a00c5ca26e4 by applying AMD Quark OCP MXFP4
quantization to the BF16 routed experts. Routed-expert weights are stored as
packed MXFP4 weights with E8M0 scales. Dense layers 0 through 2, attention,
shared experts, embeddings, norms, the audio and vision towers, MTP, and other
non-routed components remain in their source formats.
The file-to-file conversion and validation targeted AMD gfx950 and used:
docker.io/rocm/vllm-dev:nightly_main_202607142.11.0+gitd0c8b1f0.12.post1+rocm72.torch2.11The paired quality evaluation used the same GPU type and TP8 topology, with
Transformers 5.14.1 and vLLM commit
846e2d01a0be00acf31f1a354059c7c302c93042
(0.23.1rc1.dev1212+g846e2d01a).
| Benchmark | BF16 Reference | MXFP4 | MXFP4 − BF16 |
|---|---|---|---|
| BFCL exact calls | 76.54% (1,034/1,351) | 76.76% (1,037/1,351) | +0.22 pp |
| BFCL all-live macro | 76.56% | 71.01% | −5.55 pp |
| MMAU (official string match) | 75.5% (755/1,000) | 76.3% (763/1,000) | +0.80 pp |
| GPQA Diamond | 89.19% (883/990) | 87.98% (871/990) | −1.21 pp |
Refer to the Inkling-Small model card for architecture, training, intended-use, safety, and acceptable-use details.
| AIME 2026 | 94.58% (908/960) | 94.58% (908/960) | 0.00 pp |