GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
DevelopingDad
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
DevelopingDad
Model Tree
orcarouter/Qwen3.8-27B-Uncensored
Input Modalities
Output Modalities
Supported Functionality
W4A16_NVFP4, group size 16. The
exported metadata identifies 193 such target layers.kv_fp8 recipe. The audited scales are
finite, positive, and non-unit.The full export contains 2,033 indexed tensors across three safetensors
shards. ModelOpt metadata records MIXED_PRECISION weights and
kv_cache_quant_algo: FP8, produced with ModelOpt
0.47.0.dev81+ga2fbac7ba.
The KV scales were calibrated with 256 pre-rendered, text-only examples at a
2,048-token calibration sequence length, batch size 1, and
enable_thinking=false. The corpus was generic rather than application-owned:
128 general instruction/chat rows, 64 code/technical rows, 32 structured-output
rows, and 32 longer-context rows. No images or videos were used for calibration.
This establishes a scale-aware FP8-KV export for the tested distribution; it is
not a universal multimodal or application-specific calibration claim.
This artifact was validated with vllm/vllm-openai:v0.27.1. The relevant
starting flags are:
text
--quantization modelopt_fp4--kv-cache-dtype fp8_e4m3--max-model-len 262144--trust-remote-code
On the validation stack, the requested modelopt_fp4 flag resolved to vLLM's
modelopt_mixed path. The resolved KV dtype was float8_e4m3fn.
For the tested Qwen XML tool-calling route, also use:
text
--enable-auto-tool-choice --tool-call-parser qwen3_xml--default-chat-template-kwargs '{"enable_thinking":false}'
In the project's Experiment 014, the candidate returned 56/56 HTTP 200
responses with complete streaming [DONE] markers, zero stream parse errors,
and zero reasoning leaks. The explicit response-contract path passed 8/8.
The unchanged no-contract baseline passed 12/24 because the known
fenced-JSON and HH:MM formatting behaviors remained; these were output
contract misses, not cache-load failures. A short long-context load returned
24/24 complete streams at concurrency levels 1, 2, and 4; its diagnostic
fixture oracle passed 21/24. vLLM reported an allocator capacity of 2,491,134
FP8-KV tokens while the configured maximum sequence length was 262,144. The
allocator figure is a host/runtime capacity diagnostic, not a claim that the
model supports a 2.49-million-token context.
These are narrow, single-host exploratory results, not a general quality or production-readiness claim.
The Apache 2.0 license file from the derived checkpoint is included in this repository. Users are responsible for complying with the base model's terms, the derivative model's terms, and all applicable laws.