Table with columns: Tensors, Format| Tensors | Format |
|---|
| Attention, DeltaNet and MLP projections | bfloat4_b, 4.5 bits a weight; plain GPTQ |
| Joint schema head, original lexical LM table, input embedding, norms, vision encoder and every other tensor | BF16, unchanged |
Attention, DeltaNet and MLP projections use BFP4 plain GPTQ, damping 0.1, without exponent search. The
calibration uses 65,536 WikiText-2 and Korean Wikipedia tokens. quant4.json lists every projection format.
Quantized values are stored as BF16 on their block-floating grids; the archive is larger than the packed
weights. The packed estimate includes the retained vision weights and the complete BF16 joint head and lexical
table. The qualified task is text-only decision inference.
Quickstart
Use Transformers 5.23.0 and PyTorch with a CUDA device large enough for the BF16 archive values. This example
runs the original generic Cloudflare decision model, with its schema head and lexical table included.
hf download Xylolabs/clef-flash-Tensix-BFP4 --local-dir clef-flash-Tensix-BFP4cd clef-flash-Tensix-BFP4
import torchfrom joint_schema_model import collate_records, encode_record, load_release_model model, processor = load_release_model(".", device="cuda")record = { "state": {"invoice": {"vendor": "Acme", "total": 1250.0, "currency": "USD", "status": "overdue"}}, "questions": { "status": { "type": "choice", "instructions": "What is the invoice status?", "criteria": {"paid": "Invoice is paid.", "overdue": "Invoice is past due.", "draft": "Not sent."}, }, "large": {"type": "noul", "instructions": "Is the total above 1000 USD?"}, },}encoded = encode_record(processor.tokenizer, record, processor=processor)batch = collate_records([encoded], processor.tokenizer.pad_token_id, torch.device("cuda"))with torch.inference_mode(): logits = model(batch)[0]for question, question_logits in zip(encoded.questions, logits): probabilities = question_logits.float().softmax(-1).tolist() print(question.question_id, dict(zip(question.option_ids, probabilities)))
The model evaluates every allowed option in one forward pass. It does not generate a textual answer. Vision
weights are retained from the base model, but image and video inputs were not checked for this export.
Run on Tensix
All 248 quantized projections pass the grid check, native TT-NN packing and packing as BFP8. Checked with
TT-NN 0.81.1 and the Blackhole simulator; the exact example also passes. The joint schema head and lexical
table stay BF16. The quantized backbone projections use TT-NN's [in, out] matrix layout:
import jsonimport torchimport ttnnfrom safetensors import safe_open d = "." # run after downloading and entering this repositoryname = "model.language_model.layers.0.mlp.down_proj.weight"fmt = json.load(open(f"{d}/quant4.json"))["layers"]["0"]["mlp.down_proj"]["format"]dtype = {"bfp4": ttnn.bfloat4_b, "bfp8": ttnn.bfloat8_b}[fmt]shard = json.load(open(f"{d}/model.safetensors.index.json"))["weight_map"][name]with safe_open(f"{d}/{shard}", "pt") as f: w = f.get_tensor(name).t().contiguous()device = ttnn.open_device(device_id=0)w_tt = ttnn.as_tensor(w, dtype=dtype, layout=ttnn.TILE_LAYOUT, device=device, memory_config=ttnn.DRAM_MEMORY_CONFIG)assert torch.equal(ttnn.to_torch(w_tt).float(), w.float())ttnn.close_device(device)
python pack_tensix.py . checks every quantized projection; --as bfp8 checks conversion to BFP8, and
--device adds a device DRAM round trip. python verify_grid.py . checks the stored grid with torch alone.
These helpers check weight representation; they do not supply a complete Tensix decision runner.
tt-transformers and tt-forge were not checked with this package.
Evaluation
Table with columns: Decision profile, Mean option-distribution KL, Labeled right / 40, Lost labeled items| Decision profile | Mean option-distribution KL | Labeled right / 40 | Lost labeled items |
|---|
| Original BF16 backbone and head | 0 | 34 | 0 |
| This export with the original BF16 head and lexical table | 0.0084616933 | 35 | 0 |
| Same-base llama.cpp Q4_K_M backbone, original BF16 head and lexical table | 0.0061601562 | 33 | 1 |
The host decision rule-7 gate passes: KL is 1.3736 times Q4_K_M, within the 1.5-times bar, and the export
loses no additional labeled item. Every run covers the same 44 records, 55 questions and 40 labeled questions,
with records from 152 to 16,384 tokens. A loss is an originally BF16-correct labeled answer that becomes
incorrect. The KL is the mean over each question's allowed-option distribution; it is not vocabulary
language-model KL. The set includes 33 public-task questions and seven Korean questions, plus 15 unlabeled
structured-output checks.
Both reference paths use the unchanged BF16 joint schema head and the original BF16 lexical LM-head rows. The
Q4_K_M file quantizes only the backbone, converted with --no-mtp; it does not quantize the joint head. The
llama.cpp BF16 backbone floor is 0.0000711851 KL, with identical option argmaxes to the original. All
comparisons use the same original BF16 decision distributions and TF32 is disabled.
The base is Cloudflare/clef-flash at fb4417d31c1d0f9e7e77f1141b40aaede7491e00. These are host checks of the
exact saved decision profile. Speed is not reported here.
License
Apache 2.0, as the original model and generic Cloudflare code (LICENSE). Changes are documented in NOTICE;
original Xylolabs helpers use the same Apache 2.0 terms (CODE_LICENSE). Provided as is, without warranty.
Not affiliated with or endorsed by Tenstorrent.
Citation
Cite the original Cloudflare/clef-flash release.