wcamon

circus-0.4-t9

Deploy Dedicated

README

License: apache-2.0

Two artifacts, one model

model.safetensors — the quantized values materialized in bf16, so the model loads with stock transformers exactly like the base model. This is a storage/compatibility format, not a claim of 16-bit information content: every weight lies on its block's 9-level grid.
bitplanes_k2_c0.6.npz — the true quantized artifact (1.7 GB vs 8.5 GB): per target linear, 4-bit plane indices idx=(T₁+1)·3+(T₂+1), two per byte, in the GPTQ column-permuted domain, plus per-block-32 fp32 scales α and the int32 inverse column permutation inv. Scales are kept fp32 so that decoding is bit-exact against model.safetensors:
```
markdown
python decode_bitplanes.py     # verified: 200/200 tensors bit-exact
```
Raw pack ≈ 5 bits/weight (4-bit index + fp32 scale); with fp16 scales and entropy coding of the 9-way index the format reaches ≈ 3.67 bits/weight. No addition-only matmul kernel ships yet — until one exists, the bf16 materialization is how you run the model, the bit-plane pack is what the model is.

Non-target weights (embeddings, lm_head, layernorms, the vision stack) remain bf16 in both forms.

Recovery recipe (single MI325X, < 5 GPU-hours total)

GPTQ-style per-layer init with jointly solved block scales.
Sliding-window soft-anneal reconstruction (3 h): windows of 4 layers, stride 2, per-window multi-threshold tanh softening annealed s→30 over 80 % of epochs, explicit STE hard finish for the last 20 %, LoRA r=64 + multiplicative scale modulation as carriers, huber loss against the fp window's own outputs on 512 calibration segments. This puts the ternary assignment T into the gradient loop (22 % of assignments flip) — the step that end-to-end scale-only KD provably could not do in our ablations. Follows the softened-ternarization + sliding-layer reconstruction line of CAT-Q (ICML'26) / SliderQuant (ICLR'26), generalized from ternary to the additive 9-level grid.
Light polish (9 min): 50 steps of scale-only logit KD from the bf16 teacher.

Results (retention vs. bf16 base, same local harness, full test sets)

Table with columns: task, bf16, circus-0.4-t9, retention
task	bf16	circus-0.4-t9	retention
gsm8k (1319)	.8006	.7710	96.3 %
mmlu (full)	.7021	.6953	99.0 %
ifeval (541)	.2625	.2514	95.8 %
humaneval (164)	.5732	.5366	93.6 %

All ten tracked tasks retain ≥ 90 % (mbpp excluded from the headline because its train split occurs in the polish corpus). Scores are lm-eval-harness, no chat template, greedy/likelihood defaults; single seed; your numbers may differ under other harnesses.

Honest scope

At ~3.67 bits this is an easier target than pure ternary (CAT-Q) — the point of the release is the additive bit-plane form (addition-only kernels, 9/7/5 same-cost family) and the recovery methodology, not a bit-budget record.
Findings we believe transfer: (i) at this bit budget, the recovery bottleneck is assignment quality, not scales — local fp-anchored reconstruction fixes in hours what end-to-end KD cannot fix at all; (ii) val CE is not a valid intermediate judge for low-bit recovery (our best model is 0.42 nats worse in CE than a strictly weaker checkpoint); (iii) corpus-mix Pareto trade-offs observed during scale-only polishing dissolve once assignments are repaired.
Base model is a VLM; only the language stack was quantized and evaluated.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("wcamon/circus-0.4-t9", torch_dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("wcamon/circus-0.4-t9")

Citations

CAT-Q (arXiv:2606.26650) · SliderQuant (ICLR 2026) · BitNet b1.58 (arXiv:2402.17764) · ParetoQ (arXiv:2502.02631) · GPTQ (arXiv:2210.17323) · BRECQ (arXiv:2102.05426) · STE (arXiv:1308.3432) · Agents-A1 (arXiv:2606.30616)

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

wcamon

Model Tree

Base

InternScience/Agents-A1-4B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality