lued

Qwen3.8-27B-TeichAI-Fable-INT8-W8A16-MTP

README

License: apache-2.0

Relationship to the sibling quantization

This checkpoint is the Fable-5-distilled counterpart of lued/Qwen3.8-27B-INT8-W8A16-MTP. The repositories share:

the identical W8A16 compressed-tensors recipe (400 packed GEMMs, same BF16 preservation set, group-128 symmetric data-free RTN);
the identical native BF16 MTP head (15 tensors, byte-identical to Qwen's original MTP — same sha256 as the sibling release);
the identical tokenizer, chat template, processor configs, and bounded-shard layout;
the same total checkpoint footprint (29.44 GiB).

The only deviation is the base language-model weights: this checkpoint's LM weights are TeichAI's Fable-5 light-tune weights instead of stock Qwen3.8-27B.

Fable-5 distillation (inherited from the base)

Per the upstream card, the source is a light tune of Qwen/Qwen3.8-27B on the Fable-5 datasets (armand0e/claude-fable-5-claude-code, armand0e/Fable-5-Chat) plus a large corpus of private Fable 5 data, trained with Unsloth and Hugging Face TRL. Upstream-reported benchmark results versus the base: ARC Challenge 0.637 (0.591), ARC Challenge (Easy) 0.832 (0.782), BoolQ 0.911 (0.896). These are the upstream card's claims, not measurements made here. The W8A16 step touches only the 400 GEMMs; the recurrent GDN gates (96), vision tower (333 tensors), lm_head, and MTP head are preserved at source precision.

Quantization fidelity

Measured on this exact checkpoint against its own BF16 teacher (the TeichAI Fable-Distill source) on 2026-08-18 with tools/measure_kld.py (teacher-forced, full-vocabulary FP32 log-softmax, FP64 accumulation, nats/token; PyTorch 2.11.0+cu128, Transformers 5.10.1, compressed-tensors 0.17.1). This is the v0.2 revision: every preserved tensor is a bit-exact copy of the source (input_layernorm, post_attention_layernorm, q_norm/k_norm, vision tower, embeddings, lm_head, GDN gates); only the 400 projection GEMMs are quantized.

Table with columns: Candidate, Mean KLD vs BF16, Top-1 agreement, Weight files
Candidate	Mean KLD vs BF16	Top-1 agreement	Weight files
This W8 / BF16 A16	0.000700	98.29%	29.44 GiB

The KLD result measures checkpoint-weight drift over 467 teacher-forced positions (fixed 8-prompt short suite). It is not a functional quality score; tool use, coding, multimodal quality, and long-context recall require separate behavioral evaluation. A greedy KV-cache smoke generation (finite logits, non-degenerate output) passes on every release.

Short-prompt suite

Table with columns: Prompt, Category, Tokens, Positions, Mean KLD, Top-1 agreement
Prompt	Category	Tokens	Positions	Mean KLD	Top-1 agreement
`factual`	factual	13	12	0.000291	1.0000
`code-fib`	code	87	86	0.000512	0.9884

kld_result.json in this repository is the single source of truth for these numbers.

Checkpoint profile

Table with columns: Property, Value
Property	Value
Quantization	Data-free symmetric RTN W8A16, group size 128
Structural audit	`STRUCTURAL AUDIT PASS` — packed set, ignore list, index contract, value round-trip (every packed tensor dequantizes within the INT8 bound), and preserved bit-identity (all 784 non-quantized tensors byte-equal to source)
MTP audit	`MTP AUDIT PASS` (15 tensors dtype-, shape-, and bit-identical to the source)
Quantized GEMMs	400: 192 MLP, 64 full-attention, 144 dense GDN projections
Preserved precision	BF16 vision tower, `lm_head`, MTP (15 tensors, byte-identical to Qwen's original MTP), recurrent GDN gates (96)
Shard layout	7 main shards (5.34 GB max) + (849,400,392 bytes)

MTP preservation

The MTP head is the exact BF16 MTP from the original Qwen3.8-27B — never re-quantized:

15 tensors in model_mtp.safetensors (849,400,392 bytes).
sha256 1d8268aa85ace093a561e3e7b63b9d390dac1cd55a90cd55b5ec509c3c9da9fe — identical to the MTP bytes in the Qwen3.8-27B source and every family release. Verify with sha256sum model_mtp.safetensors.
All 15 tensors verified bfloat16, shape-equal, and bit-equal to the TeichAI Fable-Distill source (strict audit, not a count check); the built model_mtp.safetensors sha256 matches the family canonical hash.

Reproduction contract

The immutable source is TeichAI/Qwen3.8-27B-Fable-Distill commit ac2b133895580be05ee90a6a7cdcc23cdf998bd0. The exact selective W8A16 configuration is recorded in recipe.yaml and config.json. A conforming rebuild must take native MTP from that source, reproduce the source-derived packed projection set, and prove all 15 MTP tensors bit-identical before resharing for publication.

Status

v0.2 revision (2026-08-18). This checkpoint is built on the v0.2 pipeline: every preserved (non-quantized) tensor is a bit-exact copy of the TeichAI source — machine-audited (STRUCTURAL AUDIT PASS including the value round-trip and preserved bit-identity checks; MTP AUDIT PASS) — and KLD re-measured on the v0.2 bytes at 0.000700 nats/token, 98.29% top-1 (v0.1: 0.000660/98.50%; both within the family's measured range, and the difference is noise-scale — see the family CHANGELOG.md for the v0.1→v0.2 provenance story). Originally published 2026-08-18 at commit f41c1353d1cf238f631d42a1d0845d0471666f44 (v0.1); the v0.2 revision is a new commit on the same repo — prior commits remain fetchable by hash, so existing downloads keep working. Serving was validated on the v0.1 bytes: boots through llama-swap from the lued/* cache entry, loads the CompressedTensorsWNA16 → MarlinLinearKernel path, detects the native MTP draft model (method=mtp, 3 speculative tokens), and serves a smoke request with the qwen3 reasoning parser. The BF16 parent source remains cached; deletion is the pipeline's final step and has not been requested.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

lued

Model Tree

Base

TeichAI/Qwen3.8-27B-Fable-Distill

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality