Table with columns: Tensors, Format| Tensors | Format |
|---|
| Gated DeltaNet q/k/v, z, a, b inputs, attention q, k, v and MLP gate, up, down (32 layers) | bfloat4_b, 4.5 bits a weight |
| Gated DeltaNet and attention output projections (32 layers) | bfloat8_b, 8.5 bits a weight |
| LM head, embedding, norms | bf16 |
vision tower (model.visual) and MTP module (mtp) | bf16, as in the base model |
The quantized matrices are stored as bf16 values on their format's grid, so TT-NN packs them without changing their
values. quant4.json lists each projection's format. Only the language model was quantized and evaluated; the vision
tower and the MTP module are the base model's.
Quickstart
import torchfrom transformers import AutoModelForImageTextToText, AutoTokenizer repo = "Xylolabs/Ornith-1.5-9B-Tenstorrent-Tensix-BFP4-Out8"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForImageTextToText.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": "What is the capital of France?"}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, tokenize=True, return_tensors="pt", return_dict=True).to(model.device)with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=32, do_sample=False)print(tokenizer.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Run on Tensix
python pack_tensix.py Ornith-1.5-9B-Tenstorrent-Tensix-BFP4-Out8 packs every listed matrix with TT-NN and checks that no value changes (--device also
round-trips it through device DRAM); python verify_grid.py Ornith-1.5-9B-Tenstorrent-Tensix-BFP4-Out8 checks the grid with torch alone. Both were run on
ttsim with ttnn 0.81.1 and pass for every matrix.
Evaluation
ornith-ai/Ornith-1.5-9B
Table with columns: Weights, Formats per tensor class, Bits a weight, Size (GB), Mean KL to bf16, Top-1 (%), Answers right, Lost, Accuracy check| Weights | Formats per tensor class | Bits a weight | Size (GB) | Mean KL to bf16 | Top-1 (%) | Answers right | Lost | Accuracy check |
|---|
| bf16 (original) | every tensor bf16 | 16.00 | 17.91 | 0 | 100 | 37/40 | - | - |
| BFP4-Out8 | DeltaNet and attention output projections bfloat8_b by round to nearest; every other projection bfloat4_b by GPTQ with the exponent search; LM head and embedding bf16 | 7.35 | 8.231 | 0.0545 | 89.36 | 36/40 | 0 | passed (1.46× KL; lost 0 against 0) |
| Q4_K_M (llama.cpp) | tensor types F32 177, Q4_K 217, Q6_K 33 (plain, no imatrix) | 5.02 | 5.780 | 0.0726 | 87.44 | 36/40 | 0 | information |
| Q4_K_M-imat (llama.cpp) | tensor types F32 177, Q4_K 217, Q6_K 33 (the official ornith-ai GGUF, imatrix) | 5.02 | 5.780 | 0.0375 | 91.39 | 37/40 | 0 | reference |
All numbers are host evaluations of the weights, never device measurements; speed is not reported here. Mean KL is the KL divergence of each next-token distribution from the bf16 model's, averaged over the positions of fixed English and Korean texts; top-1 is how often both pick the same next token. The answer check is 33 fixed prompts with known answers plus 7 asked with thinking on (40 in all); an item is lost when the quantized model answers it wrong or broken where bf16 answers it right. The quantization accuracy check: a configuration passes when it loses at most one more item than llama.cpp's Q4_K_M of the same model (or the reference format named below) and its mean KL is at most 1.5 times that reference's on the same texts. Bits a weight and size count every weight read, embedding and head included. llama.cpp rows were made with llama.cpp's quantize tool from the same bf16 checkpoint. The accuracy check's reference for this model is Q4_K_M-imat, the format its maker ships.
The Q4_K_M reference is the official ornith-ai/Ornith-1.5-9B-GGUF Q4_K_M, an importance-matrix quant; the quantization
accuracy check's bar is 1.5 times its KL. These numbers measure closeness to bf16, not task quality.
License
MIT, as the base model declares. The base repository has no license file, so LICENSE holds the standard MIT text,
naming the base model's authors. Ornith-1.5 is built on Qwen3.5, whose Apache 2.0 license with its copyright notice
is in UPSTREAM_QWEN_LICENSE. The changes are listed in NOTICE. pack_tensix.py and verify_grid.py are
tensix-bfp-gptq v0.1.0's tools, unchanged, under Apache 2.0 (CODE_LICENSE). Provided as is, without
warranty. Not affiliated with or endorsed by Tenstorrent.
Citation
Cite the base model's release:
@misc{ornith_1_5, title = {{Ornith-1.5}: From Self-Scaffolding to Self-Improvement}, url = {https://ornith.ai/ornith_1_5.html}, author = {{Ornith Team}}, year = {2026}}