MingZwhy

Qwen3-0.6B-W2.79A8KV16-OPD-3phase

README

License: apache-2.0

This one loads directly

Quantization is baked into the weights (is_w_quantized: true, ~12 unique values per 128-element block) and edgerazor_config is inline in config.json.

python
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "MingZwhy/Qwen3-0.6B-W2.79A8KV16-OPD-3phase"
tok = AutoTokenizer.from_pretrained(REPO, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(REPO, trust_remote_code=True, dtype="bfloat16")

trust_remote_code=True is required, and edgerazor must be installed — weight quantization is baked in, but A8 activation and KV-cache quantization act on runtime tensors and need the code path. Without it you get an ImportError (a loud failure, unlike the latent checkpoints).

bash
pip install -e "git+https://github.com/MingZwhy/QAOPD#egg=edgerazor&subdirectory=3rdparty/edgerazor"

Results

Table with columns: Benchmark, Protocol, QAT start, this model, Qwen3-0.6B FP
Benchmark	Protocol	QAT start	this model	Qwen3-0.6B FP
GSM8K	5-shot, 1319	32.75	43.14	41.62
MATH-500	4-shot	10.40	24.20	27.20
AMC23	avg@16 (640 samples)	1.25	4.69	7.81

GSM8K exceeds the FP baseline. AMC23 recovery vs. the QAT start is significant (p = 3.9e-4).

⚠️ AMC23 must be measured at avg@16 or higher. The benchmark has 40 problems, so a single greedy pass has 2.5%/problem resolution and is unusable in this score range — we once read 12.5% at n=1 where avg@8 gave 4.06%.

⚠️ Code benchmarks are contaminated — do not quote them

The pipeline includes a deliberately contaminated stage that trains directly on MBPP-test-448 and HumanEval. Its scores (MBPP-448 169/448, HumanEval 58/164) measure fit to those evaluation sets, not generalization, and must not be reported as code-generation ability. They are published only for reproducibility of the recipe.

The math benchmarks above are clean: the training data was decontaminated against AMC23 (18 overlapping items removed) and MATH-500 (12).

How it was made

Table with columns: Phase, Data, Steps, Effect
Phase	Data	Steps	Effect
① unified math OPD	GSM8K + MATH + DAPO-17k, 2500 each	80	GSM8K 45.72 / MATH 25.20 / AMC 4.22
② contaminated code OPD	MBPP-test + HumanEval	120	code up, MATH → 19.60
③ math refresh	same as ①	60	restores MATH, lifts AMC to 4.69

Teacher: Qwen3-1.7B (bf16)
Objective: K1 sampled-token reverse KL as per-token reward, combined with GRPO policy gradient and a strict \boxed{} task reward. LR 3e-6.
Non-thinking mode. Quantization collapses this model's thinking pathway and six recovery attempts failed — see docs/LESSONS.md.

Recipe and acceptance intervals: docs/RECIPE_3PHASE.md.

Reproduction status

An independent rerun of all three phases from the shipped QAT checkpoint reproduced four of five benchmarks inside their acceptance intervals (GSM8K 44.28, MATH-500 22.80, MBPP-448 160, HumanEval 53). AMC23 came out 2.50 against 4.69 — below the interval, cause not yet established. Recorded openly in docs/RECIPE_3PHASE.md; treat the AMC figure as the least settled number here.

Quantization scheme

Table

weights	mixed int1.58/int4, block 256, 50% high-precision → 2.79 bit
activations	int8 absmax per block (256) — A8
KV cache	KV16 at deployment (trained with KV8)
embedding / lm_head	int4 absmax

License

Apache-2.0, inherited from Qwen3-0.6B.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

MingZwhy

Model Tree

Base

Qwen/Qwen3-0.6B

Fine-tuned

this model

Input Modalities

Text

Output Modalities