GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Xananthium
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Xananthium
Model Tree
HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Input Modalities
Output Modalities
Supported Functionality
I wanted W6A8, so that is what we built: six-bit stored weights, dynamic eight-bit activations, and vLLM doing the serving. Humming packs the INT6 values across INT32 word boundaries and unpacks them for the 3090's supported INT8 matrix math. The cards do not need a native six-bit Tensor Core instruction. The extra unpacking and scale work is part of the cost, and the speed tests include it.
The six-bit weights use symmetric groups of 128. Embeddings, routers, norms, vision, and designated sensitive components keep higher precision. BF16 remains the surrounding model dtype; eligible quantized linears actually use INT8 activations, with silent activation fallback disabled. TurboQuant Q4 is a separate KV-cache choice when the architecture supports it.
Accuracy matters too. A smaller checkpoint gets us memory back, but the model still has to follow the damn instructions, call tools, and get the security answers right. That is why the results include the knowledge quiz, security-policy checks, long-context recall, real Claude Code tool use, and concurrency. Failed checks stay in the report.
The measured Hauhau checkpoint shrank by 20.83%. Short-context decode changed by +5.33%, long-context decode by +1.87%, and four-request aggregate throughput by +10.19% against the tested W8A8 configuration. The exact accuracy comparison is in results/precision-comparison.json. These are local results on my two 3090s; small score differences and two timed batches do not establish a universal winner.
vLLM reported 13.57 GiB of model memory per GPU, compared with 17.32 GiB for W8A8: about 21.65% less before KV cache and graph allocations. Energy per token was not measured.
| Test | Result |
|---|---|
| CyberMetric-80-v1, primary 2048-token output budget | 79/80 |
| CyberMetric-80 after capped-response retries at 4096 tokens | 79/80 |
| Synthetic security policy checks, first answer | 27/43 |
| Synthetic long-context replay | 47/54 |
| Synthetic exact-answer reasoning, three repeated samples | 16/24 |
| Reasoning after capped-response retries at 4096 tokens | 18/24 |
| Cases correct in every repeat | 4/8 |
| Ordinary workflow recovery after an initial tool failure | 5/6 |
| Extra attempts at a failed tool approach | 8 |
| Short-context decode |
Reports in results/ contain the exact per-model profile, budgets, formatting metrics, failed checks and client integration outcomes. Counting successful test programs does not mean all accuracy checks passed. Token throughput includes reasoning tokens; it is not visible-answer throughput. The policy assessment is synthetic, and lengthy reasoning can exhaust the output budget. Eight independent long prefixes were replayed; this was not a fresh eight-turn conversation. CyberMetric is a small public knowledge quiz that may overlap training data, and does not establish practical security competence or a broad model ranking. Questions and private reasoning traces are not redistributed.
bash
hf download Xananthium/Qwen3.6-35B-A3B-Hauhau-Aggressive-W6A8-Vision --local-dir ./checkpointpip install --no-deps ./checkpoint/runtime/humming-compile-compatLD_LIBRARY_PATH="$VIRTUAL_ENV"/lib/python3.12/site-packages/nvidia/cu13/lib \LOCAL_HUMMING_COMPILE_COMPAT=1 \VLLM_HUMMING_INPUT_QUANT_CONFIG='{"dtype":"int8","group_size":0,"allow_fallback":false}' \vllm serve ./checkpoint --quantization humming --linear-backend humming --moe-backend humming --tensor-parallel-size 2 --dtype bfloat16 --trust-remote-code --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --kv-cache-dtype turboquant_4bit_nc --max-model-len 200000The loading recipe requires the architecture and quantization support in vLLM 0.31.0. Consult results/serving-profile.json for tested sampling and memory settings. Model-side INT8 activations apply only to eligible quantized linears; embeddings, routers, norms and designated sensitive components retain higher precision. KV cache precision is a separate setting. GLM MLA does not use TurboQuant in this version. Retained MTP weights do not imply speculative decoding was tested.
Seven contenders. Two RTX 3090s. The Council kept the failed checks in the record. The full report, charts, and evaluation scripts compare cybersecurity knowledge, reasoning, consistency, recovery after failed tools, long context, and concurrency. A Hugging Face mirror contains the same report.
The owner selected Nemotron Heretic W8A8 for deployment after reviewing the results. The earlier automated quality ranking remains visible and recommended REDCELL; the deployment choice does not change any measured score.
This checkpoint's complete final-round record is in results/full-round-evaluation.json. The public report explains the output-budget retries, architecture-specific cache settings, and limits of the small local comparison. No Kali workflows were run.
| 188.7645 tokens/s |
| Long-context decode | 47.281499999999994 tokens/s |
| Four simultaneous requests, aggregate | 518.582 tokens/s |