GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
just1moremodel
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
just1moremodel
Model Tree
DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
Input Modalities
Output Modalities
Supported Functionality
| Quant method | Quark (quant_method: quark) |
| Weight format | MXFP4 (fp4, group size 32, E8M0 scales), U8-packed |
| Activation format | fp8 (dynamic) — W4A8 |
| AWQ | Enabled — per-layer scaling search over all 64 layers (MLP scaling map from AMD's Qwen3.8-27B-Quark-AWQ-MXFP4 reference) |
self_attn | Quantized (U8-packed, matching AMD's reference — no attention exclusion) |
| Excluded (BF16) | lm_head + full vision tower (111 model.visual.* entries) |
| MTP / drafter head | 8 mtp.* projections requantized BF16 → FP8-E4M3 per-channel weights + FP32 scales (declared in layer_quant_config), for speculative decoding |
| Tensors | 1703 total: 1346 language_model.* + 23 mtp.* + 333 visual.* + lm_head |
| Architecture | Qwen3_5ForConditionalGeneration (multimodal wrapper; vision tower intact in BF16) |
| Params | 27B dense, 64 layers |
Per the source card: a multi-stage tuned model combining Cold Fusion 735/882 ratio training, GAIN training, and an uncensored "NM/DAU" heretic treatment on the Qwen3.8-27B base — aimed at instruction following, creative prose, and unrestricted assistant behavior. Refer to the source model card for full details on the training method and intended use. This repo only changes the precision (BF16 → MXFP4/AWQ); weights are otherwise derived from that source.
Standard OpenAI-compatible server:
bash
vllm serve /path/to/Qwen3.8-27B-TURBO-Fable-MXFP4-awq \--host 0.0.0.0 --port 8000 \--tensor-parallel-size 2 \--max-model-len 32768 \--gpu-memory-utilization 0.90 \--trust-remote-code
Notes:
quant_method: quark support should load it via its Quark integration — if your build lacks MXFP4 kernels for your GPU, you will get a kernel-availability error at load, not silent slowdown.chat_template.jinja included) with reasoning-effort control (xhigh/medium/low, default xhigh) and XML tool calling. Pass chat_template_kwargs: {"enable_thinking": false} to disable thinking mode.--speculative-config '{"method":"mtp","num_speculative_tokens":5}' works on stacks that support Qwen MTP drafting.quant_method: quark, fp4/group-32 weights + activations, exclude = lm_head + vision, self_attn U8-packed, 18.04 GiB (matches AMD's reference size for this architecture — no accidental exclusions)algo_config nulled (vLLM quark mapper compatibility), FP8 MTP declaredUncensored model — intended for adults, research, and responsible use; no built-in safety alignment. All limitations of the source apply (see its card). Quantization (MXFP4 W4A8) adds minor degradation vs BF16; AWQ scaling was calibrated to minimize output deviation on general text, but has not been benchmarked against the BF16 source with formal metrics.