Table with columns: Tensors, Format| Tensors | Format |
|---|
| attention q, k, v, o (10 layers) | bfloat4_b, 4.5 bits a weight |
| Gated DeltaNet q, k, v, z, a, b, output (30 layers) | bfloat4_b, 4.5 bits a weight |
| routed experts gate, up, down and shared expert (40 layers) | bfloat4_b, 4.5 bits a weight |
| shared expert gate (40 layers) | bf16 |
| LM head, embedding, norms and every other tensor | bf16, from the dequantized source |
4.84 bits a weight on average, 20.99 GB packed. quant4.json lists each projection's format. Quantized with
GPTQ and a per-block exponent search, calibrated on 65,536 tokens of WikiText-2 and Korean Wikipedia. The
routed experts' gate and up share one Hessian per layer; each expert's down takes its own, or uses round to
nearest when it saw too few calibration tokens.
Text only: the source GGUF holds the language model, so this release has no vision tower (load it with
AutoModelForCausalLM). It has no MTP layer either, as the source has none. For speculative decoding the base
model's MTP layer can serve as the drafter: the verify step keeps this model's own tokens.
Quickstart
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "Xylolabs/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Tenstorrent-Tensix-BFP4"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16", device_map="auto") messages = [{"role": "user", "content": "What is the capital of France?"}]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=64)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Run on Tensix
Checked with ttnn 0.81.1 on ttsim, Tenstorrent's Blackhole simulator. Pack one matrix and
check that no value changed (TT-NN reads weights as [in, out]):
import jsonimport torchimport ttnnfrom safetensors import safe_open d = "Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Tenstorrent-Tensix-BFP4" # hf download Xylolabs/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Tenstorrent-Tensix-BFP4 --local-dir Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Tenstorrent-Tensix-BFP4name = "model.language_model.layers.0.linear_attn.in_proj_a.weight"fmt = json.load(open(f"{d}/quant4.json"))["layers"]["0"]["linear_attn.in_proj_a"]["format"]dtype = {"bfp4": ttnn.bfloat4_b, "bfp8": ttnn.bfloat8_b}[fmt]shard = json.load(open(f"{d}/model.safetensors.index.json"))["weight_map"][name]with safe_open(f"{d}/{shard}", "pt") as f: w = f.get_tensor(name).t().contiguous() device = ttnn.open_device(device_id=0)w_tt = ttnn.as_tensor(w, dtype=dtype, layout=ttnn.TILE_LAYOUT, device=device, memory_config=ttnn.DRAM_MEMORY_CONFIG)assert torch.equal(ttnn.to_torch(w_tt).float(), w.float())ttnn.close_device(device)
python pack_tensix.py Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Tenstorrent-Tensix-BFP4 checks every matrix this way
(--device also round-trips it through device DRAM).
python verify_grid.py Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Tenstorrent-Tensix-BFP4
checks the grid with torch alone. The same values also
pack exactly into
ttnn.bfloat8_b.
tt-metal and tt-forge. Not run with this checkpoint.
Evaluation
Table with columns: Bits a weight, Size (GB), Mean KL to the reference, Top-1 (%), Answers right, Lost | Bits a weight | Size (GB) | Mean KL to the reference | Top-1 (%) | Answers right | Lost |
|---|
| reference: the source's Q8_K_P, dequantized to BF16 | 16 | 69.32 | 0 | 100 | 40 of 40 | - |
| this model | 4.84 | 20.991 | 0.04970 | 89.90 | 40 of 40 | 0 |
| Q4_K_M (llama.cpp, from the reference, no importance matrix) | 4.88 | 21.167 | 0.04213 | 90.69 | 40 of 40 | 0 |
| HauhauCS's own Q4_K_M (with an importance matrix), for comparison | 4.88 | 21.167 | 0.02706 | 92.48 | 40 of 40 | 0 |
KL and top-1 cover 8,188 positions of English and Korean text (WikiText-2 test, Korean Wikipedia), as plain
text. Answers are 33 short prompts with automatic checks, plus the 7 reasoning ones again with thinking on (40
in all). "Lost" means right in the reference but wrong or broken here; near ties are not counted. Our Q4_K_M was
made from the reference with llama.cpp v0.9.0 (cd959bae) and no importance matrix, as for every model we
release; its sha256 is 91a2a71997ce46bb3d16360615a8580a1210b2b82ab0154bcaa688dfc72b77fd. Our bar: mean KL at
most 1.5 times that Q4_K_M's and at most one more lost answer. Passed: 1.18 times its KL, 0 lost against 0.
HauhauCS's Q4_K_M (sha256 bbef58c37ce88820be9d98b6437f1cf4bac890c947bd55fc7b68e22098574231) is closer to the
reference: against it this model's KL is 1.84 times.
These numbers measure closeness to the reference, not task quality or safety. Speed is not reported here.
License
Apache 2.0, as the base model and the source (LICENSE). The changes are listed in NOTICE. pack_tensix.py and
verify_grid.py are by Xylolabs under the same license. Provided as is, without warranty. Not affiliated with or
endorsed by Tenstorrent, the Qwen team or HauhauCS.
Citation
@misc{qwen36_35b_a3b, title = {{Qwen3.6-35B-A3B}: Agentic Coding Power, Now Open to All}, url = {https://qwen.ai/blog?id=qwen3.6-35b-a3b}, author = {{Qwen Team}}, month = {April}, year = {2026}}