Table with columns: Tensors, Format| Tensors | Format |
|---|
| Gated DeltaNet q/k/v, z, a, b inputs and output (48 layers) | bfloat4_b, 4.5 bits a weight |
| attention q, k, v, o (16 layers) | bfloat4_b, 4.5 bits a weight |
| MLP gate, up, down (64 layers) | bfloat4_b, 4.5 bits a weight |
| LM head, embedding, norms | bf16 |
vision tower (model.visual) and MTP module (mtp) | bf16, as in the base model |
linear_attn.A_log and linear_attn.norm.weight (48 layers) | float32, as in the base model |
The quantized matrices are stored as bf16 values on the BFP4 grid, so TT-NN packs them without changing their
values. quant4.json lists each projection's format. Quantized with GPTQ and a per-block exponent search, calibrated
on 65,536 tokens of WikiText-2 and Korean Wikipedia. Only the language model was quantized and evaluated; the vision
tower and the MTP module are the base model's.
Quickstart
import torchfrom transformers import AutoModelForImageTextToText, AutoTokenizer repo = "Xylolabs/Qwen3.5-27B-Tenstorrent-Tensix-BFP4"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForImageTextToText.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": "What is the capital of France?"}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, tokenize=True, return_tensors="pt", return_dict=True).to(model.device)with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=32, do_sample=False)print(tokenizer.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Run on Tensix
python pack_tensix.py Qwen3.5-27B-Tenstorrent-Tensix-BFP4 packs every listed matrix with TT-NN and checks that no value changes (--device also
round-trips it through device DRAM); python verify_grid.py Qwen3.5-27B-Tenstorrent-Tensix-BFP4 checks the grid with torch alone. Both were run on
ttsim with ttnn 0.81.1 and pass for every matrix. The same values also pack exactly into ttnn.bfloat8_b.
Evaluation
Qwen/Qwen3.5-27B
Table with columns: Weights, Formats per tensor class, Bits a weight, Size (GB), Mean KL to bf16, Top-1 (%), Answers right, Lost, Accuracy check| Weights | Formats per tensor class | Bits a weight | Size (GB) | Mean KL to bf16 | Top-1 (%) | Answers right | Lost | Accuracy check |
|---|
| bf16 (original) | every tensor bf16 | 16.00 | 53.79 | 0 | 100 | 39/40 | - | - |
| BFP4 | every linear layer (attention, DeltaNet and MLP) bfloat4_b by GPTQ with the exponent search; head bf16; the rest bf16 (vision and MTP left out) | 5.59 | 18.788 | 0.0254 | 93.69 | 39/40 | 0 | passed (1.33× KL; lost 0 against 0) |
| Q4_K_M (llama.cpp) | tensor types F32 353, Q4_K 433, Q6_K 65 (plain, no imatrix) | 4.92 | 16.547 | 0.0192 | 94.38 | 39/40 | 0 | reference |
All numbers are host evaluations of the weights, never device measurements; speed is not reported here. Mean KL is the KL divergence of each next-token distribution from the bf16 model's, averaged over the positions of fixed English and Korean texts; top-1 is how often both pick the same next token. The answer check is 33 fixed prompts with known answers plus 7 asked with thinking on (40 in all); an item is lost when the quantized model answers it wrong or broken where bf16 answers it right. The quantization accuracy check: a configuration passes when it loses at most one more item than llama.cpp's Q4_K_M of the same model (or the reference format named below) and its mean KL is at most 1.5 times that reference's on the same texts. Bits a weight and size count every weight read, embedding and head included. llama.cpp rows were made with llama.cpp's quantize tool from the same bf16 checkpoint.
These numbers measure closeness to bf16, not task quality. Speed is not reported here.
License
Apache 2.0, as the base model (LICENSE, copied from it). The changes are listed in NOTICE. pack_tensix.py and
verify_grid.py are tensix-bfp-gptq v0.1.0's tools, unchanged, under Apache 2.0 (CODE_LICENSE).
Provided as is, without warranty. Not affiliated with or endorsed by Tenstorrent.
Citation
@misc{qwen3.5, title = {{Qwen3.5}: Towards Native Multimodal Agents}, author = {{Qwen Team}}, month = {February}, year = {2026}, url = {https://qwen.ai/blog?id=qwen3.5}}