The package contains 1,952 indexed tensors. All 105 active text MLP gate, up and down matrices use BFP4-valued
weights; all 100 active attention q, k, v and o matrices use BFP8-valued weights. Per-layer projections, token
and per-layer lookup tables, the tied head and norms remain BF16. Inactive original encoder tensors and an
explicit tied-head copy remain for checkpoint completeness.
Table with columns: Quantity, Value| Quantity | Value |
|---|
| Original saved export | 11,046,518,689 bytes |
| Active text values | 4,628,569,379 |
| Logical packed active text weights | 6,757,759,558 bytes |
| Average bits per active text value | 11.680 |
| Packed selected matrices plus all other stored tensors | 8,514,524,102 bytes |
The large BF16 lookup tables make the whole-text average higher than the matrix formats alone. Safetensors
preserve the BF16-valued quantized weights for framework loading; the included helpers check their BFP4 and
BFP8 packing.
Quickstart
Use a Transformers release that supports Gemma 4. This exact text-only framework example is checked with
Transformers 5.22.1:
import torchfrom transformers import AutoModelForImageTextToText, AutoTokenizer repo = "Xylolabs/gemma-4-E2B-it-Tenstorrent-Tensix-BFP4-MLP-Cal4x"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForImageTextToText.from_pretrained( repo, dtype=torch.bfloat16, device_map="cpu",)messages = [{"role": "user", "content": "What is 2 + 2?"}]inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, enable_thinking=False, tokenize=True, return_tensors="pt", return_dict=True,)with torch.no_grad(): output = model.generate(**inputs, max_new_tokens=32, do_sample=False)print(tokenizer.decode(output[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Framework loading uses the saved BF16-valued tensors, totaling about 11.05 GB before runtime buffers. The
included pack_tensix.py and verify_grid.py check the declared compressed formats. Host TT-NN packing requires
a configured architecture context, such as an upstream simulator environment; its format checks do not qualify
a complete native model runner.
Evaluation
The quantization accuracy check on the saved files passes on the host: 29/33 normal answers versus BF16’s 30/33, and 6/7 thinking
answers versus BF16’s 7/7. One answer counts as lost, within the accuracy check's allowance of one more than Q4_K_M’s
zero. The counted normal loss is en-instr-lines. The thinking item en-reason-order is a near tie, which
the frozen scorer excludes. Q4_K_M answers 30/33 normal and 7/7 thinking items with no counted loss.
Table with columns: Saved-file check, Result| Saved-file check | Result |
|---|
| Original Q4_K_M chat KL | 0.29329 |
| This checkpoint chat KL | 0.128496 |
| KL ratio to Q4_K_M | 0.438 |
| Normal answers right | 29/33 |
| Thinking answers right | 6/7 |
| Counted lost or broken answers | 1 |
The original 32 evaluation sequences and 16,144 scored positions are read back. The accuracy check's KL uses the
original 16 chat sequences, 7,968 positions and all 262,144 vocabulary entries. All 33 normal and seven thinking prompt IDs,
generation budgets, stop IDs and frozen scoring rules remain unchanged. Stored BF16 log-probability values
match their original PT sources without a new reference forward.
All 205 selected matrices pass the exact grid check and TT-NN host packing round trips in BFP4 and BFP8. The
other 1,747 tensors, the tied head included, match the pinned source exactly. These format checks use an
offline upstream architecture context. They qualify the preserved weights and their formats. Native execution
requires compatible support and its own validation. Vision and audio task performance are not evaluated here.
Speed is not reported here.
License
Apache 2.0, as declared by the pinned Google release. The primary repository supplies no root legal file;
LICENSE contains the complete standard Apache Software Foundation text. NOTICE identifies the source, changes
and scope. Original Xylolabs helper code uses Apache 2.0 (CODE_LICENSE). Provided as is, without warranty or
upstream endorsement. Not affiliated with or endorsed by Tenstorrent. pack_tensix.py and verify_grid.py are tensix-bfp-gptq v0.1.0's tools, unchanged.
Citation
Cite the original Google Gemma 4 E2B Instruct release and its
recommended technical report:
@misc{gemmateam2026gemma4, title={Gemma 4 Technical Report}, author={Gemma Team}, year={2026}, eprint={2607.02770}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2607.02770}, }