tcclaviger

Tess-27B-RFI

README

License: apache-2.0

Quantization by component

Attention and all other linear layers — 8-bit integer weights (RFI): group size 32, symmetric, Hadamard-32 rotation, block-float scales stored as int8 mantissa + int8 exponent, int8 activation compute path.
MTP speculative-decode head — also 8-bit RFI: its fc, MLP, and attention projections are all int8-packed like the main attention; only its norms stay bf16.
MLP layers — 4-bit float weights (RFA): IQ4_NL non-linear grid, group size 16, asymmetric, Hadamard-16 rotation, the same block-float int8 scale encoding.
Kept in bf16 (not quantized) — vision encoder, linear-attention (GDN) blocks, embeddings, norms, and the lm_head.

Serving context — 512K via YaRN

All evaluation below was run while serving at --max-model-len 524288 (512K tokens), extended from the native 256K window with YaRN via --hf-overrides:

json
{"text_config": {"rope_parameters": {"rope_type": "yarn", "factor": 2.0,
 "original_max_position_embeddings": 262144, "mrope_interleaved": true,
 "mrope_section": [11, 11, 10], "partial_rotary_factor": 0.25,
 "rope_theta": 10000000}}}

Evaluation results

Six builds measured with identical methodology, each against its own live vLLM endpoint (July 2026): Qwen3.6-27B (bf16 base) → Tess-4-27B (the tune, bf16) → Tess-27B-RFI (this quant) → Tess-27B-RFA (all-attention 4-bit sibling) → Tess-FP8 (W8A8 block-128 FP8 sibling), with Qwen3.6-35B-A3B (MoE, bf16) as a comparative reference point. Bold marks the best score in each row (ties all bolded).

Tune impact — Qwen3.6-27B → Tess-4-27B

Table with columns: Metric, Change
Metric	Change
WikiText-2 perplexity	7.056 → 6.669 (−5.5%)
Codeneedle overall recall	97.8% → 97.7% (≈ flat)
MC accuracy (4 tasks)	≈ flat (−0.1 to +1.6 pp)
Tool-eval (full 69, TC-61 excl)	86 → 85 (−1)
GSM8K / MMLU (50q each)	98→94% / 74→76%
Decode @ conc 1 (ISL 128)	57.0 → 59.9 tok/s (+5%)
Decode @ conc 50 (ISL 128)	556 → 528 tok/s (−5%)

Quantization cost (RFI/RFA hybrid) — Tess-4-27B → Tess-27B-RFI

Table with columns: Metric, Change
Metric	Change
Checkpoint size	55.6 → 28.9 GB (−48%)
WikiText-2 perplexity	6.669 → 6.663 (≈ flat)
Codeneedle overall recall	97.7% → 98.0% (≈ flat)
MC accuracy (4 tasks)	≈ flat (−0.4 to +0.1 pp)
Tool-eval (full 69, TC-61 excl)	85 → 87 (+2)
GSM8K / MMLU (50q each)	94→98% / 76→82% (both up)
Decode @ conc 1 (ISL 128)	59.9 → 75.3 tok/s (+26%)
Decode @ conc 50 (ISL 128)	528 → 453 tok/s (−14%)

Quality

Table with columns: Metric, Qwen3.6-27B (base), Tess-4-27B, Tess-27B-RFI, Tess-27B-RFA, Tess-FP8, Qwen3.6-35B-A3B
Metric	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
WikiText-2 PPL (n_ctx 2048, lower is better)	7.0559	6.6691	6.6632	6.6292	6.6627	6.5092
ARC-Challenge (acc_norm)	59.30%	60.84%	60.41%	60.32%

Multiple-choice accuracy is lm-eval loglikelihood scoring, 0-shot.

Long-context positional recall (codeneedle)

Verbatim function recall under 10K–80K-token contexts.

Table with columns: Corpus, Qwen3.6-27B (base), Tess-4-27B, Tess-27B-RFI, Tess-27B-RFA, Tess-FP8, Qwen3.6-35B-A3B
Corpus	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
Python	100%	100%	100%	100%	99.55%	99.09%
C++	98.12%	98.12%	98.44%

Tool calling & accuracy benches

Table with columns: Bench, Qwen3.6-27B (base), Tess-4-27B, Tess-27B-RFI, Tess-27B-RFA, Tess-FP8, Qwen3.6-35B-A3B
Bench	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
tool-eval final (full 69, TC-61 excl)	86	85	87	86	87	90
GSM8K (50q)	98.0%	94.0%	98.0%

Decode throughput — tok/s output (ISL 128 / ISL 512)

vllm bench serve, random dataset, OSL 128, saturation, 4× R9700 (gfx1201), TP 4.

Table with columns: Concurrency, Qwen3.6-27B (base), Tess-4-27B, Tess-27B-RFI, Tess-27B-RFA, Tess-FP8, Qwen3.6-35B-A3B
Concurrency	Qwen3.6-27B (base)	Tess-4-27B	Tess-27B-RFI	Tess-27B-RFA	Tess-FP8	Qwen3.6-35B-A3B
1	57.0 / 61.4	59.9 / 63.8	75.3 / 69.3	67.9 / 58.2	86.0 / 85.1	91.9 / 114.4
10	304.5 / 233.6	289.6 / 227.8	281.5 / 219.2	292.2 / 194.8

MTP draft acceptance by work category

Measured from live serving logs, k=5 draft tokens, drafted-token-weighted aggregation.

Table with columns: Work category, Overall acceptance, Pos 1, Pos 2, Pos 3, Pos 4, Pos 5
Work category	Overall acceptance	Pos 1	Pos 2	Pos 3	Pos 4	Pos 5
JSON generation	79.0%	92.5%	85.9%	79.3%	71.6%	65.6%
Math	78.5%	94.4%	87.4%	78.2%	70.7%

Notes

All builds serve on the tcclaviger/vllm:latest image, which has kernel tunes baked in. TunableOp is untuned — GEMMs run on default heuristic-determined values. Base Qwen3.6-27B figures are the 2026-07-12 re-measurement on the same tcclaviger/vllm:latest image and thinking-OFF methodology as every other build, replacing an earlier non-comparable run.

Credits

Tess-4-27B by Migel Tissera (migtissera/Tess-4-27B) — the model quantized here:

bibtex
@misc{tissera2026tess4,
  title        = {Tess-4-27B},
  author       = {Migel Tissera},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/migtissera/Tess-4-27B}}
}

codeneedle (positional recall) originally by Alexander Ziskind, expanded test suite by tcclaviger (Rob Smith).
Tool-calling scenarios (incl. TC-61) run on tool-eval-bench by SeraphimSerapis (Tim Messerschmidt), scenario methodology adapted from ToolCall-15 by stevibe.
GSM8K, MMLU, and IFEval run via tool-eval-bench's built-in accuracy benchmarks at their defaults: GSM8K 8-shot CoT, MMLU 5-shot, IFEval zero-shot.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

tcclaviger

Model Tree

Base

migtissera/Tess-4-27B

Quantized

this model

Input Modalities

Text

Image

Video

Output Modalities