Table with columns: Tensors, Format| Tensors | Format |
|---|
| attention q, k, v, o (8 layers) | bfloat8_b, 8.5 bits a weight |
| Gated DeltaNet q, k, v, z, a, b, output (24 layers) | bfloat8_b, 8.5 bits a weight |
| MLP gate, up, down (32 layers) | bfloat8_b, 8.5 bits a weight |
| LM head, embedding, norms and every other tensor | bf16, except the original dtypes below |
mtp, visual | bf16, unchanged |
Original tensors: model.language_model.layers.N.linear_attn.A_log, model.language_model.layers.N.linear_attn.norm | float32, unchanged |
10.21 bits a weight on average, 11.42 GB packed. quant4.json lists each projection's format. Quantized with
round to nearest on the bfloat8_b grid, each block of 16 taking the shared exponent of its largest value or
one lower, whichever gives the smaller error; no calibration.
Only the language model was quantized and evaluated.
Quickstart
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "Xylolabs/Qwen3.5-9B-Tenstorrent-Tensix-BFP8-RTN-Exp"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16", device_map="auto") messages = [{"role": "user", "content": "What is the capital of France?"}]inputs = tok.apply_chat_template(messages, enable_thinking=False, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=64)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
For CPU use, install Transformers without the optional GPU-only flash-linear-attention and causal-conv1d
packages so its PyTorch fallbacks are used.
Run on Tensix
Checked with ttnn 0.81.1 on ttsim, Tenstorrent's Blackhole simulator. Pack one matrix and check that no value
changed (TT-NN reads weights as [in, out]):
import jsonimport torchimport ttnnfrom safetensors import safe_open d = "Qwen3.5-9B-Tenstorrent-Tensix-BFP8-RTN-Exp" # hf download Xylolabs/Qwen3.5-9B-Tenstorrent-Tensix-BFP8-RTN-Exp --local-dir Qwen3.5-9B-Tenstorrent-Tensix-BFP8-RTN-Expname = "model.language_model.layers.0.mlp.down_proj.weight"fmt = json.load(open(f"{d}/quant4.json"))["layers"]["0"]["mlp.down_proj"]["format"]dtype = {"bfp4": ttnn.bfloat4_b, "bfp8": ttnn.bfloat8_b}[fmt]shard = json.load(open(f"{d}/model.safetensors.index.json"))["weight_map"][name]with safe_open(f"{d}/{shard}", "pt") as f: w = f.get_tensor(name).t().contiguous() device = ttnn.open_device(device_id=0)w_tt = ttnn.as_tensor(w, dtype=dtype, layout=ttnn.TILE_LAYOUT, device=device, memory_config=ttnn.DRAM_MEMORY_CONFIG)assert torch.equal(ttnn.to_torch(w_tt).float(), w.float())ttnn.close_device(device)
python pack_tensix.py Qwen3.5-9B-Tenstorrent-Tensix-BFP8-RTN-Exp checks every matrix this way (--device also
round-trips it through device DRAM). python verify_grid.py Qwen3.5-9B-Tenstorrent-Tensix-BFP8-RTN-Exp checks the grid
with torch alone.
tt-transformers and tt-forge were not run with this checkpoint.
Evaluation
This is a preserved-weight export. A retained host-reference comparison of these saved text weights reports
mean KL 0.001096 to BF16 over 16,312 positions of English and Korean text (WikiText-2 test and Korean
Wikipedia), plain and in the chat template. These figures describe distribution closeness to the BF16
reference.
No answer-check verdict is retained for this export. The device answer check has not been run on these exact
weights, and no rule-7 verdict is claimed. Packing checks do not establish complete model inference accuracy.
Speed is not reported here.
License
Apache 2.0, as the base model (LICENSE). The changes are listed in NOTICE. pack_tensix.py and
verify_grid.py are by Xylolabs under the same license. Provided as is, without warranty. Not affiliated with
or endorsed by Tenstorrent.
Citation
@misc{qwen3.5, title = {{Qwen3.5}: Towards Native Multimodal Agents}, author = {{Qwen Team}}, month = {February}, year = {2026}, url = {https://qwen.ai/blog?id=qwen3.5}}