GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
IteraCompute
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
IteraCompute
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
| Item | Details |
|---|---|
| Model | Qwen3.8-27B |
| Quantization | NVIDIA Model Optimizer NVFP4; see config.json and hf_quant_config.json for layer coverage |
| Weight files | Two Safetensors shards, 17,916,112,584 bytes in total (about 17.92 GB) |
| Configuration architecture | Qwen3_5ForConditionalGeneration / qwen3_5, as supplied upstream |
| Configured maximum context | 262,144 tokens, input and output combined |
| IteraCompute deployment | vLLM 0.31.0, one RTX 5090 32 GB per instance, FP8 KV cache |
| License | Apache 2.0; retain LICENSE and upstream notices |
NVFP4 execution requires compatible Blackwell hardware and engine support. Available context and concurrency depend on engine settings and memory. The checkpoint's context configuration is separate from the limits of any hosted API plan. This release does not add a new quality evaluation or benchmark; figures in the preserved source card belong to its authors and their stated test conditions.
bash
hf download IteraCompute/Qwen3.8-27B-NVFP4 --local-dir ./Qwen3.8-27B-NVFP4cd Qwen3.8-27B-NVFP4sha256sum -c SHA256SUMSFor reproducible use, add --revision <full-commit-sha> with a commit from this repository's history. SHA256SUMS covers the checkpoint, serving files, and preserved upstream documentation. release.json records the source commit and confirms comparison with the deployed checkpoint.
The following settings reflect IteraCompute's vLLM 0.31.0 deployment. Install an appropriate CUDA/Blackwell build of vLLM in a separate environment, then run:
bash
vllm serve ./Qwen3.8-27B-NVFP4 \ --served-model-name qwen3.8-27b-nvfp4 \ --host 127.0.0.1 --port 8000 \ --quantization modelopt \ --tensor-parallel-size 1 \ --max-model-len 262144 \ --gpu-memory-utilization 0.97 \ --max-num-seqs 16 \ --max-num-batched-tokens 2048 \ --kv-cache-dtype fp8 \ --attention-backend FLASHINFER \ --enable-chunked-prefill \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser qwen3_xml \ --reasoning-parser qwen3bash
curl http://127.0.0.1:8000/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{"model":"qwen3.8-27b-nvfp4","messages":[{"role":"user","content":"Hello!"}],"max_tokens":256,"chat_template_kwargs":{"enable_thinking":false}}'The example binds to localhost. Engine and driver compatibility must be checked for your environment. Use the included chat template for reasoning and tool calls; further upstream details are in SOURCE_MODEL_CARD.md.
IteraCompute plans to upload a Qwen 4 NVFP4 version as soon as possible after the official model becomes available and licensing, quantization, and inference validation are complete. Follow our Qwen4-27B-NVFP4 repository for the release. That repository is currently a preparation page with no Qwen 4 weights; its name is a planned target, subject to the official model lineup.
中文：我们后续也会第一时间推进并上传 Qwen 4 的 NVFP4 版本。 将在官方模型可用、确认许可证并完成量化及推理验证后尽快发布。预告仓库已准备好：IteraCompute/Qwen4-27B-NVFP4。目前尚未提供 Qwen 4 权重，最终型号以官方发布为准。
IteraCompute provides OpenAI-compatible model inference. Visit our website, API documentation, and current free Qwen API offering for hosted access and current terms.
This is an independent community distribution, unaffiliated with Qwen or NVIDIA. Original model and quantization credit remains with the respective upstream authors.
hf download IteraCompute/Qwen3.8-27B-NVFP4 --local-dir ./Qwen3.8-27B-NVFP4cd Qwen3.8-27B-NVFP4sha256sum -c SHA256SUMSvllm serve ./Qwen3.8-27B-NVFP4 \ --served-model-name qwen3.8-27b-nvfp4 \ --host 127.0.0.1 --port 8000 \ --quantization modelopt \ --tensor-parallel-size 1 \ --max-model-len 262144 \ --gpu-memory-utilization 0.97 \ --max-num-seqs 16 \ --max-num-batched-tokens 2048 \ --kv-cache-dtype fp8 \ --attention-backend FLASHINFER \ --enable-chunked-prefill \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser qwen3_xml \ --reasoning-parser qwen3curl http://127.0.0.1:8000/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{"model":"qwen3.8-27b-nvfp4","messages":[{"role":"user","content":"Hello!"}],"max_tokens":256,"chat_template_kwargs":{"enable_thinking":false}}'