GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
RedHatAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
RedHatAI
Model Tree
thinkingmachines/Inkling
Input Modalities
Output Modalities
Supported Functionality
This model is intended for deployment with vLLM. You can serve the model using 4xB200s:
bash
vllm serve RedHatAI/Inkling-NVFP4-FP8-BLOCK \--tokenizer-mode inkling \--tensor-parallel-size 4 \--enable-auto-tool-choice \--tool-call-parser inkling \--reasoning-parser inkling \--enable-expert-parallel \--max-model-len 100000 \--max-num-seqs 32 \--enable-chunked-prefill \--trust-remote-code \
Evaluations were performed with inspectai:
bash
inspect eval hf/Idavidrein/gpqa/diamond \--model RedHatAI/Inkling-NVFP4-FP8-BLOCK \--reasoning-effort high \--model-base-url http://localhost:8000/v1 \
| Benchmark | thinkingmachines/Inkling | RedHatAI/Inkling-NVFP4-FP8-BLOCK |
|---|---|---|
| GPQA Diamond | 87.2 | 84.5 |
Note: A bug in inspectai currently misscores markdown-formatted multiple choice answers (e.g. ANSWER: $B$ rather than ANSWER: B), which the Inkling models often include in their response. Scores are manually evaluated to account for discrepancy. More information at https://github.com/UKGovernmentBEIS/inspect_ai/issues/5145