GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
darrellbest
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
darrellbest
Model Tree
darrellbest/Qwen3.8-27B-Heretic
Input Modalities
Output Modalities
Supported Functionality
| Part | Precision |
|---|---|
| Linear layers of the MLPs and attention | FP8 (float8_e4m3fn, dynamic per-token activation scales) |
Gated DeltaNet (linear_attn) layers, vision tower, embeddings, lm_head, MTP | bf16 (unchanged) |
The linear-attention layers carry a recurrent state and the vision tower encodes images; both stay in bf16, as in
the FP8 builds of related models. Multi-token-prediction weights are preserved in model-auxiliary.safetensors
(save_pretrained drops them for this architecture).
Made with llm-compressor 0.13.0, FP8_DYNAMIC, which needs no
calibration data.
sh
vllm serve darrellbest/Qwen3.8-27B-Heretic-FP80/100 refusals against 98/100 for the original, KL divergence 0.0465. Method and parameters are on the parent card. The FP8 weights themselves were not re-measured; the figures are the parent model's.
Reduced safety guardrails by design. You are responsible for what you do with it.
| Repository | Format | Size | Use it with |
|---|---|---|---|
| Qwen3.8-27B-Heretic | bf16 safetensors | 51 GB | transformers, vLLM, SGLang |
| Qwen3.8-27B-Heretic-GGUF | GGUF BF16 / Q8_0 / Q4_K_M (+ vision) | 51 / 27 / 16 GB | llama.cpp, Ollama |
| Qwen3.8-27B-Heretic-FP8 | FP8 W8A8, compressed-tensors |
vllm serve darrellbest/Qwen3.8-27B-Heretic-FP8| 35 GB |
| vLLM |
| Qwen3.8-27B-Heretic-NVFP4 | NVFP4, compressed-tensors | 27 GB | vLLM on Blackwell |