Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-MXFP4 API & Inference Endpoint

Packed (496 modules → weight_packed + weight_scale): all 64 dense mlp.{gate,up,down}_proj, all 16 full-attention self_attn.{q,k,v,o}_proj, and all 48 linear_attn.{in_proj_qkv,in_proj_z,out_proj,in_proj_a,in_proj_b} — quantized with the inference kernel's own quantizer (torch.ops.vllm.xpu_mxfp4_quantize — format match by construction).
BF16: embed, lm_head, all norms, A_log/conv1d/dt_bias, the full 27-block visual tower, and the MTP draft block (mtp.*, merged from the source's separate mtp-bf16.safetensors, reference layout).
text_config.mtp_num_hidden_layers is 1 (as shipped upstream).

1,695 tensors / 16 shards / 18.44 GiB. SHA256SUMS included.

Measured quality of the quantization

In-conversion spot checks (kernel dequantizer, 1-in-64, shard 1): max mean rel-err 0.127, mean 0.120 — on the ~0.12 gate of the Tiel/KAT/Occamy MoE builds.

Measured serving (Intel Arc 140V, vLLM 0.30.0+xpu, eager)

Dense 27B is memory-bandwidth bound here (expect ~3 tok/s, same wall as the Solstice dense-Qwen3.8 reference). Example launch (text tower only):

bash

vllm serve Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-MXFP4 \  --served-model-name Huihui-Qwen3.8-27B \  --max-model-len 4096 --max-num-seqs 1 --gpu-memory-utilization 0.68 \  --language-model-only --enforce-eager

Provenance

Reference schema: Solstice-AI/Qwen3.8-27B-TURBO-Fable-...-NM-DAU-MXFP4 (dense MXFP4, verified loadable on this box). Built with quantize_huihui_qwen38_mxfp4.py (see alpha2beta/vllm.xpu). Base weights: huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated (BF16, Apache-2.0).

Packed (496 modules → weight_packed + weight_scale): all 64 dense mlp.{gate,up,down}_proj, all 16 full-attention self_attn.{q,k,v,o}_proj, and all 48 linear_attn.{in_proj_qkv,in_proj_z,out_proj,in_proj_a,in_proj_b} — quantized with the inference kernel's own quantizer (torch.ops.vllm.xpu_mxfp4_quantize — format match by construction).
BF16: embed, lm_head, all norms, A_log/conv1d/dt_bias, the full 27-block visual tower, and the MTP draft block (mtp.*, merged from the source's separate mtp-bf16.safetensors, reference layout).
text_config.mtp_num_hidden_layers is 1 (as shipped upstream).

1,695 tensors / 16 shards / 18.44 GiB. SHA256SUMS included.

Measured quality of the quantization

In-conversion spot checks (kernel dequantizer, 1-in-64, shard 1): max mean rel-err 0.127, mean 0.120 — on the ~0.12 gate of the Tiel/KAT/Occamy MoE builds.

Measured serving (Intel Arc 140V, vLLM 0.30.0+xpu, eager)

Dense 27B is memory-bandwidth bound here (expect ~3 tok/s, same wall as the Solstice dense-Qwen3.8 reference). Example launch (text tower only):

bash

vllm serve Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-MXFP4 \  --served-model-name Huihui-Qwen3.8-27B \  --max-model-len 4096 --max-num-seqs 1 --gpu-memory-utilization 0.68 \  --language-model-only --enforce-eager

Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-MXFP4

README

Contents

Measured quality of the quantization

Measured serving (Intel Arc 140V, vLLM 0.30.0+xpu, eager)

Provenance

Explore FriendliAI today

README

Contents

Measured quality of the quantization

Measured serving (Intel Arc 140V, vLLM 0.30.0+xpu, eager)

Provenance