GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
ghostchoir
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ghostchoir
Model Tree
Qwen/Qwen3-1.7B
Input Modalities
Output Modalities
Supported Functionality
| bit-width | 3 |
| granularity | per-channel (--w_group_size -1) |
| scheme | asymmetric, weight-only (activations stay bf16) |
| CBQ settings | paper defaults |
| CBD window / overlap | 2 / 1 |
| epochs per window | 3 |
| LoRA-Rounding rank | 5 |
| CFP weight truncation | tensor |
Delta_W hard-round fraction | 0.1 |
| held-out perplexity | 11.2810 (bf16 baseline 2.0084, delta +9.2726) |
| mean per-token KL(FP || quant) | 1.85642 over 63406 tokens |
Calibration: 4096 rows of open-r1/OpenR1-Math-220k rendered with the Qwen3 chat template and truncated to 2048 tokens. This is not the paper's calibration set (the paper uses 128 samples); it is matched to a sibling QAT framework so the two are comparable.
Perplexity above is on held-out rows of the same distribution and is a sanity metric only — it is not a reasoning-benchmark score.
Produced by an unofficial CBQ implementation written from the paper text; CBQ has
no official code release. The paper is ambiguous or self-inconsistent in several
places (notably the interaction of Eq. 8 and Eq. 11 for Delta_W initialisation, and
the direction of the Algorithm 1 objective), and every such choice is documented with
the measurement behind it. Two settings differ from the paper deliberately:
group-wise rather than per-channel quantization where noted, and the calibration set
above.
Not affiliated with the CBQ authors or with Qwen. Inherits the base model's license.