GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
ghostchoir
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ghostchoir
Model Tree
Qwen/Qwen3-1.7B
Input Modalities
Output Modalities
Supported Functionality
| bit-width | 4 |
| granularity | group-128 (--w_group_size 128) |
| scheme | asymmetric, weight-only (activations stay bf16) |
| CBQ settings | group-tuned (see below) |
| CBD window / overlap | 2 / 1 |
| epochs per window | 3 |
| LoRA-Rounding rank | 5 |
| CFP weight truncation | off |
Delta_W hard-round fraction | 1.0 |
| held-out perplexity | 2.1147 (bf16 baseline 2.0084, delta +0.1063) |
| mean per-token KL(FP || quant) | 0.05527 over 63406 tokens |
Calibration: 4096 rows of open-r1/OpenR1-Math-220k rendered with the Qwen3 chat template and truncated to 2048 tokens. This is not the paper's calibration set (the paper uses 128 samples); it is matched to a sibling QAT framework so the two are comparable.
Perplexity above is on held-out rows of the same distribution and is a sanity metric only — it is not a reasoning-benchmark score.
Produced by an unofficial CBQ implementation written from the paper text; CBQ has
no official code release. The paper is ambiguous or self-inconsistent in several
places (notably the interaction of Eq. 8 and Eq. 11 for Delta_W initialisation, and
the direction of the Algorithm 1 objective), and every such choice is documented with
the measurement behind it. Two settings differ from the paper deliberately:
group-wise rather than per-channel quantization where noted, and the calibration set
above.
This checkpoint uses --cfp_weight off and
--hard_round_frac 1.0 rather than the paper's values. Both paper settings were
measured worse at group-128: CFP weight truncation costs ~+2.6 perplexity because
at group-128 an outlier only inflates its own 128-wide group, and rounding
Delta_W only in the late phase leaves most of the optimization on a near-lossless
(degenerate) objective.
Not affiliated with the CBQ authors or with Qwen. Inherits the base model's license.