GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
leoncca
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
leoncca
Model Tree
huihui-ai/Huihui-Qwen3.8-27B-abliterated
Input Modalities
Output Modalities
Supported Functionality
The source is pinned to
huihui-ai/Huihui-Qwen3.8-27B-abliterated@d42ca8978c5a66e92c3446d46e8adfe03ef692ff.
The included manifests and validation artifacts describe the exact build.
Validation used 4x Tesla V100-PCIE-32GB, TP4, a 262,144-token window, FP16
compute, FP16 KV, and 1Cat-vLLM v0.1.dev17347+gc8a1a528e (main commit
c8a1a528). Long requests were confirmed on all four ranks to use the SM70
D=256 Split-D and direct-paged prefill paths rather than a silent fallback.
| Evaluation | BF16 source | Mixed FP8 |
|---|---|---|
| Core text, JSON, tool, and real-image checks | 10/10 | 10/10 |
| Needle retrieval at 8K, 64K, 128K, 246K | 4/4 | 4/4 |
| GSM8K fixed 32-example subset | 28/32 | 28/32 |
| HumanEval + MBPP functional execution | 9/10 | 9/10 |
| IFEval strict/loose prompts | 3/5 | 3/5 |
| IFEval strict/loose instructions |
The public47 set had 35/47 exact answers and no correctness flip. Across 5,046 shared-prefix selected tokens, the mean absolute log-probability delta was 0.004285 (median 0.000008). All four long-retrieval answers were exact matches.
On the same bounded sequential public47 run, total request time was 232.30 s for BF16 and 171.66 s for mixed FP8. Long, prefill-dominated retrieval was 1.7%-2.4% slower for mixed FP8. These are paired validation observations, not a general throughput claim.
MTP0 is the validated and recommended profile for this release. The native MTP weights are preserved in BF16, but MTP4 was not accepted as part of this checkpoint's runtime contract. FP8 KV was also intentionally excluded because long-context quality was prioritized over additional memory savings.
The production-validated core arguments are:
bash
vllm serve leoncca/Qwen3.8-27B-Huihui-Mixed-FP8 \--trust-remote-code \--dtype float16 \--attention-backend FLASH_ATTN_V100 \--tensor-parallel-size 4 \--gpu-memory-utilization 0.92 \--kv-cache-dtype auto \--max-model-len 262144 \--max-num-seqs 4 \--max-num-batched-tokens 15680 \--enable-prefix-caching \--mamba-cache-mode align \--enable-auto-tool-choice \--tool-call-parser qwen3_coder \--reasoning-parser qwen3
On this stack, --kv-cache-dtype auto selected FP16 KV because the runtime
compute dtype was FP16. Confirm the selected dtype and optimized-kernel route in
your own startup logs.
The source model describes itself as abliterated and uncensored. A narrow,
harmless ALLOW/REFUSE proxy was also run without eliciting harmful details;
both the BF16 source and this FP8 derivative returned REFUSE for all six proxy
categories. This does not establish a universal refusal rate or safety profile,
but it means the upstream branding alone should not be treated as proof of a
zero-refusal model.
SHA256SUMS covers every published file other than itself. The release includes
the pinned source manifest, precision manifest, per-tensor FP8 error metrics,
static audit, and bounded validation summaries.
This release inherits the capabilities and risks of the Huihui source. It may produce inaccurate, unsafe, illegal, or otherwise harmful content. Users are responsible for prompts, outputs, downstream actions, access control, legal compliance, and deployment-appropriate safeguards. The weights are provided as-is without warranty.
Apache-2.0, inherited from the source checkpoint and Qwen base model.
| 8/12 |
| 9/12 |