GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
MingZwhy
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
MingZwhy
Model Tree
Qwen/Qwen3-0.6B
Input Modalities
Output Modalities
Supported Functionality
The tensors in this repo are bf16 and NOT quantized. QAT keeps high-precision master weights and applies fake-quant only inside the forward pass; what gets saved is the master copy.
python
# This gives you an UNQUANTIZED model. It will not error, and it will score higher# than the real 1.88-bit model.AutoModelForCausalLM.from_pretrained("MingZwhy/Qwen3-0.6B-W1.88A8KV8-QAT-step9000")
config.json here is a plain Qwen3ForCausalLM with no auto_map and no
edgerazor_config, so nothing in the file tells the loader to quantize. To use it as
a 1.88-bit model you must supply both the EdgeRazor code and the exact quantization
config it was trained under.
Want a model you can just load and evaluate? Use Qwen3-0.6B-W2.79A8KV16-OPD-3phase (the W2.79 sibling) instead — quantization is baked in there.
This form is exactly what you want for further QAT/OPD, because the optimizer needs the continuous master weights. The training scripts pass the quantization config themselves:
bash
git clone https://github.com/MingZwhy/QAOPD && cd QAOPDbash tools/install_fresh_env.shSTUDENT_MODEL=<this checkpoint> bash scripts/train/run_unified_math.sh
| weights | weight_quant_uniform_symmetric_clip_per_block_mp_int1_58_int4_static_row_wise_sparse — mixed int1.58/int4, block 256, 12.5% high-precision → 1.88 bit effective |
| activations | int8 absmax per block (block 256) — A8 |
| KV cache | int8, block 128 (exported as KV16 for deployment) |
| embedding / lm_head | int4 absmax override |
| distillation | KD logits loss (alpha 2.0, T=1.0) + task loss (alpha 0.1), teacher Qwen3-0.6B FP |
EdgeRazor QAT distillation on general instruction data (ii/tulu/am/task mixtures with teacher-regenerated assistant turns), step 9000. Full procedure: docs/EDGERAZOR_QAD.md.
The 3-phase recovery pipeline has been validated on the W2.79 sibling checkpoint; W1.88 is the next planned run and has no recovery numbers yet. Expect larger degradation at this bit-width. See docs/ROADMAP.md.
Do not assume the W2.79 recipe transfers unchanged — the project's own rule is that every model/bit-width combination needs its checkpoint sweep redone.
Apache-2.0, inherited from Qwen3-0.6B.