Table with columns: Tensors, Format| Tensors | Format |
|---|
| attention q, k, v, o (10 layers) | bfloat4_b, 4.5 bits a weight |
| Gated DeltaNet q, k, v, z, a, b, output (30 layers) | bfloat4_b, 4.5 bits a weight |
| routed experts gate, up, down and shared expert (40 layers) | bfloat4_b, 4.5 bits a weight |
| shared expert gate (40 layers) | bf16 |
| LM head, embedding, norms and every other tensor | bf16, as in the base model |
mtp, visual | bf16, unchanged |
4.84 bits a weight on average, 20.99 GB packed. quant4.json lists each projection's format. Quantized with
GPTQ and a per-block exponent search, calibrated on 65,536 tokens of WikiText-2 and Korean Wikipedia. The
routed experts' gate and up share one Hessian per layer; each expert's down takes its own, or uses round to
nearest when it saw too few calibration tokens.
Only the language model was quantized and evaluated.
Quickstart
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "Xylolabs/Qwen3.6-35B-A3B-Tensix-BFP4"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16", device_map="auto") messages = [{"role": "user", "content": "What is the capital of France?"}]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=64)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Run on Tensix
Tested with ttnn 0.81.1 on ttsim, Tenstorrent's Blackhole simulator; not on hardware. Pack one matrix and
check that no value changed (TT-NN reads weights as [in, out]):
import jsonimport torchimport ttnnfrom safetensors import safe_open d = "Qwen3.6-35B-A3B-Tensix-BFP4" # hf download Xylolabs/Qwen3.6-35B-A3B-Tensix-BFP4 --local-dir Qwen3.6-35B-A3B-Tensix-BFP4name = "model.language_model.layers.0.linear_attn.in_proj_a.weight"fmt = json.load(open(f"{d}/quant4.json"))["layers"]["0"]["linear_attn.in_proj_a"]["format"]dtype = {"bfp4": ttnn.bfloat4_b, "bfp8": ttnn.bfloat8_b}[fmt]shard = json.load(open(f"{d}/model.safetensors.index.json"))["weight_map"][name]with safe_open(f"{d}/{shard}", "pt") as f: w = f.get_tensor(name).t().contiguous() device = ttnn.open_device(device_id=0)w_tt = ttnn.as_tensor(w, dtype=dtype, layout=ttnn.TILE_LAYOUT, device=device, memory_config=ttnn.DRAM_MEMORY_CONFIG)assert torch.equal(ttnn.to_torch(w_tt).float(), w.float())ttnn.close_device(device)
python pack_tensix.py Qwen3.6-35B-A3B-Tensix-BFP4 checks every matrix this way (--device also round-trips
it through device DRAM). python verify_grid.py Qwen3.6-35B-A3B-Tensix-BFP4 checks the grid with torch alone.
The same values also pack exactly into ttnn.bfloat8_b.
tt-metal Qwen3.5/3.6 demo (v0.81.1). tt-transformers does not support this architecture. tt-metal's
Qwen3.5/3.6 demo runs
Qwen3.6-35B-A3B tensor-parallel on four Blackhole P150s, not on one chip. It packs the DeltaNet and attention
projections in bfloat8_b, the gate and up of the experts and the shared expert in bfloat4_b, and their down
projections in bfloat8_b. Every projection of layers 0 to 3 packed exactly, experts included. That was checked
on one simulated chip; the four-chip split was not run. The head, which this checkpoint keeps in bf16, is
re-quantized, because the demo packs it in bfloat8_b. Its MTP block is off by default for this model.
# build tt-metal v0.81.1 as the demo's README describes, thenexport HF_MODEL=Xylolabs/Qwen3.6-35B-A3B-Tensix-BFP4export MESH_DEVICE=P150x4pytest models/demos/blackhole/qwen36/demo/text_demo.py -v -s -k "traced_128 and not traced_128k"
tt-forge (tt-xla). Not run with this checkpoint yet.
Evaluation
Table with columns: Bits a weight, Size (GB), Mean KL to BF16, Top-1 (%), Answers right, Lost | Bits a weight | Size (GB) | Mean KL to BF16 | Top-1 (%) | Answers right | Lost |
|---|
| BF16 (Qwen/Qwen3.6-35B-A3B) | 16 | 69.32 | 0 | 100 | 39 of 40 | - |
| this model | 4.84 | 20.991 | 0.05013 | 89.62 | 39 of 40 | 0 |
| Q4_K_M (llama.cpp) | 4.89 | 21.713 | 0.0404 | 90.80 | 39 of 40 | 0 |
KL and top-1 cover 8,188 positions of English and Korean text (WikiText-2 test, Korean Wikipedia), as plain
text. Answers are 33 short prompts with automatic checks, plus the 7 reasoning ones again with thinking on (40
in all). "Lost" means right in BF16 but wrong or broken here; near ties are not counted. Q4_K_M was made from
the same checkpoint with llama.cpp v0.9.0 (cd959bae) and no importance matrix; its sha256 is
bb73f0f0e6f8004816edfb0bf51280473b6f05dfca0a12fbc85ae5aea1d061ec. Our bar: mean KL at most 1.5 times
Q4_K_M's and at most one more lost answer. Passed: 1.24 times Q4_K_M's KL, 0 lost against 0.
These numbers measure closeness to BF16, not task quality; no device speed was measured. For the model itself,
see the base card.
License
Apache 2.0, as the base model (LICENSE). The changes are listed in NOTICE. pack_tensix.py and
verify_grid.py are by Xylolabs under the same license. Provided as is, without warranty. Not affiliated with
or endorsed by Tenstorrent.
Citation
@misc{qwen36_35b_a3b, title = {{Qwen3.6-35B-A3B}: Agentic Coding Power, Now Open to All}, url = {https://qwen.ai/blog?id=qwen3.6-35b-a3b}, author = {{Qwen Team}}, month = {April}, year = {2026}}