BlairQ

qwen3.8_27b_idiotSavant_sm_86

README

License: apache-2.0

TL;DR

Table

What	orcarouter/Qwen3.8-27B-Uncensored (BF16, 55 GB) → 18.6 GB, int4 weights (symmetric, group 128), residual stream rotated
Served as	W4A8: int4 weights × int8 activations on the tensor cores (Marlin, vLLM)
Quality	KL to BF16 on assistant tokens: 0.0178, vs 0.0385 for the popular AutoRound int4 → −54%. On the Qwen3.8-27B card benchmarks it ties BF16 on short reasoning and loses on long reasoning and fine vision (Benchmarks)
Speed	Same as unrotated int4. Prefill 2368 tok/s at 28k, 2014 at 57k; decode ~130 tok/s on prose with one request and the drafter (2×3090, capped, 2026-10-10)
Drafter	qwen3.8_27b_idiotSavant_sm_86_dflash2: DFlash2, re-tuned against this model, 5.62 accepted tokens/step on agentic coding
Needs	vLLM + mandatory online Hadamards: 512-block before `down_proj`, one per head before `o_proj` / `out_proj` (Genesis PN148 + PN154; see Serving)
Weights	2026-09-28, updated in place: per-head Hadamard on `o_proj`/`out_proj` and GDN gates `in_proj_a/b` in int4 (KL 0.0194 → 0.0178). This card: 2026-10-10

🧰 Serving stack: everything this card relies on — the vLLM patches, the integer kernels, the tree speculation and the exact compose it is measured with — lives in blairq/genesis-vllm-patches. Known-good setup: compose/docker-compose.qwen38-27b-idiotsavant-sm86.yml.

⚠️ This is not a drop-in int4 checkpoint. The weights live in a rotated basis: down_proj expects its input multiplied by a 512-block Hadamard, and o_proj / out_proj by a Hadamard of one head (256 / 128). Without them the model outputs garbage. Small hooks are enough (see below).

Benchmarks against the BF16 model card

Every benchmark from the Qwen3.8-27B card that can be run with an endpoint and a public dataset, run against this model as served (W4A8, int8 KV, DFlash2 tree speculation, 2× RTX 3090). Official prompts and graders, the card's sampling (thinking, temperature=1.0, top_p=0.95, top_k=20), reasoning_effort=xhigh, one sample per item. Finished 2026-10-10; HLE and LiveCodeBench then re-run with a 200k-token cap (the first run used 81 920).

Table with columns: Benchmark, idiotSavant, Qwen3.8-27B BF16 (card), Δ, n
Benchmark	idiotSavant	Qwen3.8-27B BF16 (card)	Δ	n
IFBench (prompt-level loose)	83.0 ±4.3	79.5	+3.5	300
GPQA Diamond	91.9 ±3.8	89.2	+2.7	198
HLE (fixed sample)	23.6 ±3.7 (28.2 closing `</think>`¹)	30.8

± is the 95% interval of the local score. ¹ Closing the </think> that 81 HLE answers never emitted; Genesis PN175 now does it at serving time (see below).

How to read it.

Short reasoning: a tie. IFBench, GPQA and ERQA sit inside their intervals: no visible loss from the quantization. With enough token budget LiveCodeBench ties too (90.8 vs 90.3).
Long reasoning and fine perception: real losses. HLE and BabyVision lose.
What goes wrong in long reasoning, measured.
- Truncation: with the first 81 920-token cap, 16% of HLE and 10 of 131 LiveCodeBench answers were cut while still thinking. With 200k only 8 of 500 HLE answers hit the cap, and LiveCodeBench went from 85.5 to 90.8. The card budgets up to 262k tokens of reasoning.
- Unclosed reasoning: 16% of HLE answers end without emitting </think>; the answer is written inside the reasoning and the parser drops it. The rate grows with length: 0% below 8k tokens, 8% at 16–32k, 25% at 64–82k.
- Grading those answers anyway gives HLE 28.2. Forcing the close instead (append </think> and let the model write the answer) gets 23 of those 81 right, against 24 by grading the reasoning. That is what Genesis PN175 does at serving time. Whether the BF16 does the same at this effort is not measured.

Not an apples-to-apples comparison.

Different reference model. The BF16 column is Qwen's own number for the official model. This one is quantized from orcarouter's abliterated BF16, so the gap mixes abliteration, quantization and harness differences.
Judge. HLE, CharXiv and BabyVision are judged by this same model, not by GPT-4o as in the card.
Smaller splits for HLE and MathVision (see n).
Not reproduced: the agentic rows of the card (SWE-bench Pro, Terminal Bench, OSWorld, …) need their own environments.

Suite and raw results: benchmarks/qwen38_card and summary.md (200k re-run).

Why "idiot savant"

On Ampere (sm_86) the tensor cores run fp16 1×, int8 4× and int4 8×. The plan was to compute in int8 across the whole decode: int4 weights (decode is memory-bound), int8 per-token activations.

The catch nobody mentions: int8 per-token activations hurt as much as int4 weights. Each token's int8 scale is set by its max, and the inputs that come off the residual stream have a crest factor (max/rms) of ~56–70. One massive-activation channel (#3994) crushes everything else:

Table with columns: linear, int8 per-token error, no rotation, worst layers, crest
linear	int8 per-token error, no rotation	worst layers	crest
k_proj / v_proj	10–14%	37–48% (3, 7, 11, 15)	~56
in_proj_qkv (GatedDeltaNet)	12%	24% (6)	~58
gate / up	5–7%	12%	~32
down_proj	6%	11%	~27

Rotating the residual stream with a randomized block-Hadamard (QuaRot-style) spreads that spike across all channels: crest ~70 → ~2.7, int8 error <1% on every linear. The rotation is baked into the weights, so it costs nothing at runtime. As a bonus, GPTQ quantizes the rotated weights better: int4 error drops in all 64 layers (median −8%, layer 0 −43%).

Results

KL(BF16 ‖ model) on assistant-generated positions of real agentic-coding traffic (opencode with tools), top-20 support. Measuring on prompt text is meaningless: a chat model predicts <|im_end|> at ~99% on user and system turns, and any perturbation flips it.

Table with columns: torch W4A16, torch W4A8, vLLM W4A8 (as served), top-1
	torch W4A16	torch W4A8	vLLM W4A8 (as served)	top-1
noon AutoRound int4 (popular)	0.0155	0.0300	0.0385	0.938
GPTQ, not rotated	0.0122	0.0289	0.0365	0.940
idiotSavant, 2026-09-25	0.0104	0.0116	0.0194	0.956

Without rotation, int8 activations wipe out GPTQ's advantage over AutoRound. With rotation, int8 activations cost only +12% KL.

Speed on 2× RTX 3090, TP=2, power-capped (229 / 243 W), vLLM 0.29.0. Rotation against no rotation, measured back to back on 2026-09-25:

Table with columns: prefill 28k, prefill 57k, decode (1 req, no spec.)
	prefill 28k	prefill 57k	decode (1 req, no spec.)
GPTQ, not rotated	2699 tok/s	2293 tok/s	~58 tok/s
idiotSavant (fused kernel)	2721 tok/s	2298 tok/s	~60 tok/s

Rotation costs nothing. With today's serving stack (2026-10-10: cold prompt, max_tokens=1) prefill is 2368 tok/s at 28k and 2014 at 57k. That is 9% below 2026-09-28 (2590 / 2180); it comes from serving-stack changes, not from the weights, and is still being bisected.

With the DFlash2 drafter and tree speculation, on a 40-request agentic bank (4 concurrent, 4 server boots, 2026-10-03): 5.62 accepted tokens/step. The 2026-09-25 measurement with the previous drafter, 3 replicas, was 5.63 and ~216 tok/s.

What was done, layer by layer

Row convention y = x·Wᵀ. R = diag(±1 signs, seed 20260924) · block-Hadamard(1024). g = 1 + w is the GemmaRMSNorm weight, folded into the next linear (the norm itself becomes w = 0).

Table with columns: where, stored weight, Hessian for GPTQ, why
where	stored weight	Hessian for GPTQ	why
q/k/v, in_proj_qkv/z, gate/up	`W·diag(g)·Rᵀ`	`R·H_n·Rᵀ`	they read the residual: that is where the spike lives
o_proj, out_proj	`R·W·Hc`	`Hc·H·Hc`	they write the residual; their input (crest 12–21) is the only one not rotated, so `Hc` = Hadamard of one head (256 attention, 128 GDN), applied online
down_proj

Quantization: GPTQ, int4 symmetric, group 128, fp16 scales = max|w|/7.5, damp 0.01, no act-order (Marlin reads contiguous groups).
Calibration: 256 × 4096 tokens of real agentic-coding traffic, tokenized with the served chat template.
Layer 7 gotcha: it has a norm channel with g = 0 exactly, on the massive-activation channel. After rotation that energy leaks into the MLP unless the Hessian is taken on the pre-norm-weight input n. Done here.
MTP head: dropped. The drafter is DFlash2, and the MTP head would be invalid in the rotated basis.

The full rationale, every alternative that was measured, and the numbers behind each decision are in reproducir/DECISIONES.md.

Serving

vLLM (tested: v0.29.0, 2× RTX 3090, TP=2)

The hard requirements are the Hadamards: before down_proj (PN148, fused with SiluAndMul and the int8 quantization, SK-23) and per head before o_proj / out_proj (PN154, fused with the output gate / RMSNormGated and the int8 quantization, SK-25). With PN155 the int4 GDN gates run inside the in_proj_qkvz Marlin. All three come with the Genesis patches, and config.json lists them in genesis_rotacion.requiere:

bash

GENESIS_ENABLE_PN148_ROT_DOWN=1       # mandatoryGENESIS_ENABLE_PN154_HAD_SALIDAS=1    # mandatory (per-head Hadamard before o_proj / out_proj)GENESIS_ENABLE_PN155_BA_EN_QKVZ=1     # GDN gates in int4 inside the in_proj_qkvz MarlinGENESIS_ENABLE_PN149_ROT_BORRADOR=1   # only if you use the DFlash2 drafterVLLM_MARLIN_INPUT_DTYPE=int8          # W4A8 (without it: W4A16, also works) vllm serve BlairQ/qwen3.8_27b_idiotSavant_sm_86 \  --quantization compressed-tensors --dtype float16 --tensor-parallel-size 2 \  --kv-cache-dtype int8_per_token_head --max-model-len 262144 \  --speculative-config '{"method":"dflash","model":"BlairQ/qwen3.8_27b_idiotSavant_sm_86_dflash2","num_speculative_tokens":8}'

The complete compose this runs with, every setting explained next to its measurement: compose/docker-compose.qwen38-27b-idiotsavant-sm86.yml. As of 2026-10-10 it adds, on top of the mandatory patches:

8-node tree speculation with the DFlash2 drafter;
int8 KV (int8_per_token_head) read by an integer PTX decode kernel, and an int8 P2P all-reduce;
prefix cache that survives multi-turn agent work: sparse GDN checkpoints plus the reuse boundary (PN168), and an L2 KV offload to RAM that actually hits on this hybrid model with speculation (PN170/PN171). On a replayed real opencode session (threads of ~100K and ~125K plus a sub-agent) 70% of prompt tokens come from cache and every follow-up turn is 93–97% cached; a 100K turn evicted from the GPU comes back from RAM with TTFT 55.9 s → 4.3 s. The L2 is 8 GiB since 2026-10-10 (was 12), and PN174 keeps a crashed engine from leaving it behind in /dev/shm;
the multimodal processor cache at 1 GiB instead of vLLM's 4 GiB, which is duplicated in the API server and the engine and filled a 26 GB container until OOM;
scheduling for agents: one prefill at a time with priority bypasses, and a dynamic prefill chunk so a turn that is generating is not slowed to the pace of another thread's prefill;
tool-calling fixes for long contexts.

Details and measurements: the repo README.

Anything else (transformers, other engines)

Apply the Hadamard to the input of every mlp.down_proj (block 512), of every self_attn.o_proj (block 256 = one head) and of every GDN out_proj (block 128 = one head). Everything else is a standard compressed-tensors checkpoint:

python

import torch def block_hadamard(x, b=512):    h = torch.ones(1, 1, dtype=torch.float64)    while h.shape[0] < b:        h = torch.cat([torch.cat([h, h], 1), torch.cat([h, -h], 1)], 0)    H = (h / b ** 0.5).to(x.device, x.dtype)    return (x.reshape(*x.shape[:-1], -1, b) @ H).reshape(x.shape) HAD = {"mlp.down_proj": 512, "self_attn.o_proj": 256, "linear_attn.out_proj": 128}   # block = one head for o/out_proj for layer in model.model.language_model.layers:          # after from_pretrained(...)    for name, b in HAD.items():        try:            mod = layer.get_submodule(name)              # attention layers have o_proj, GDN layers out_proj        except AttributeError:            continue        mod.register_forward_pre_hook(lambda m, a, b=b: (block_hadamard(a[0], b),) + a[1:])

This exact hook is what validated the checkpoint in torch (KL 0.0104 A16 / 0.0112 simulated A8). It has not been tested through from_pretrained end-to-end. The rotation parameters are in config.json under genesis_rotacion.

Reproduce

Anyone can rebuild these exact weights from the BF16 with the reproducir/ folder. It contains the pipeline, the exact calibration set, a local virtualenv setup, and a guide (reproducir/README.md):

bash

hf download BlairQ/qwen3.8_27b_idiotSavant_sm_86 --include "reproducir/*" --local-dir idiotsavantcd idiotsavant/reproducirbash reconstruir.sh --dry-run        # venv + downloads the BF16 (55 GB) + validates everything (~20 s)setsid nohup bash reconstruir.sh > reconstruir.log 2>&1 &              # ~85 min on 2× RTX 3090bash correr.sh idiotsavant.py estado --trabajo trabajo --salida qwen3.8_27b_idiotSavant_sm_86   # progress (--json for scripts)

Isolated: the virtualenv and every cache stay inside the folder, so nothing is installed system-wide.
Resumable: calibration and quantization run as separate processes, one per GPU, and relaunching resumes what was done.
Resource-aware: it checks RAM (including cgroup limits), GPU and disk, and waits instead of crashing.
Other calibration: calibracion.py builds your own from the public source dataset (--desde-hf) or from your own chat logs (--conversaciones).

The weights in this repo are exactly the output of that script with the bundled calibration (256 × 4096 tokens); by default it builds the current weights (2026-09-28), and ARMADO=2509 builds the previous ones. The same folder is entrenamiento/idiotsavant/ in genesis-vllm-patches. The per-layer report is in informe_idiotsavant.json: local error of each served layer, int4 error per linear, crest before and after rotation, and dead channels. The script was validated against the original research build: 98.6% identical int4 values, served KL 0.0193 vs 0.0194, same top-1.

Limitations

Uncensored. It inherits the abliteration of the base model: refusals were removed. You are responsible for how you use it.
Tuned and measured for agentic coding with tools on sm_86. It runs elsewhere, but the numbers above are only for that.
It needs the online Hadamards (see above), so it is not a drop-in int4 checkpoint.
Long reasoning degrades. Beyond ~16k tokens of thinking it sometimes ends without closing </think>, and at xhigh effort it can run past an 81 920-token budget (HLE: 12% and 16% of the answers). Give it a large max_tokens, or a lower reasoning_effort, for hard problems.
Vision. The vision tower is kept as-is in bf16, but the language model that reads its tokens is int4. It holds up on charts, documents and real-world photos (CharXiv −2.4, RealWorldQA −3.9) and loses most on fine visual discrimination (BabyVision −19.6).

Credits and references

Base model: Qwen/Qwen3.8-27B, abliterated by orcarouter.
Methods: GPTQ (Frantar et al., 2022) · QuaRot (arXiv 2404.00456) · SpinQuant (2405.16406) · MambaQuant (2501.13484) · KL-lens sensitivity for hybrid models (2604.13440).
Kernels and serving: vLLM and Marlin; the Genesis patches for vLLM (blairq/genesis-vllm-patches, forked from Sandermage/genesis-vllm-patches).
prompts derived from (CC-BY-4.0), shipped tokenized in .

Resumen en español

Qwen3.8-27B sin censura, comprimido a int4 con el residuo rotado, para calcular en int8 en las RTX 3090. La rotación de Hadamard baja la cresta de las activaciones de ~70 a ~2,7, y con eso el int8 por token deja de romper la calidad: la mitad de error que el int4 de AutoRound, a la misma velocidad. Para servirlo hacen falta las Hadamard antes de down_proj (Genesis PN148) y por cabeza antes de o_proj / out_proj (PN154); con PN155 las compuertas del GDN van en int4 dentro del Marlin de in_proj_qkvz. Pesos actualizados en su lugar el 2026-09-28: KL 0,0194 → 0,0178.

Contra el BF16 en los benchmarks del card de Qwen (2026-10-10, ver la tabla de arriba):

Empata en razonamiento corto: IFBench 83,0 contra 79,5; GPQA 91,9 contra 89,2; ERQA 67,5 contra 65,5.
LiveCodeBench empata con tope de 200k tokens: 90,8 contra 90,3 (con 81 920 daba 85,5).
Pierde en HLE: 23,6 contra 30,8 con tope de 200k; 28,2 si se cierra el </think> que falta.
Pierde en percepción fina: BabyVision 46,1 contra 65,7.
Lo que falla en el razonamiento largo es que el modelo termina sin cerrar </think>, y eso pasa más cuanto más largo es el razonamiento. Genesis PN175 lo cierra al servir y el modelo responde.

Todas las decisiones, con sus mediciones, están en reproducir/DECISIONES.md. Los parches, el compose con el que se sirve y la suite de benchmarks: genesis-vllm-patches.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

BlairQ

Model Tree

Base

orcarouter/Qwen3.8-27B-Uncensored

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

TL;DR

Table

What	orcarouter/Qwen3.8-27B-Uncensored (BF16, 55 GB) → 18.6 GB, int4 weights (symmetric, group 128), residual stream rotated
Served as	W4A8: int4 weights × int8 activations on the tensor cores (Marlin, vLLM)
Quality	KL to BF16 on assistant tokens: 0.0178, vs 0.0385 for the popular AutoRound int4 → −54%. On the Qwen3.8-27B card benchmarks it ties BF16 on short reasoning and loses on long reasoning and fine vision (Benchmarks)
Speed	Same as unrotated int4. Prefill 2368 tok/s at 28k, 2014 at 57k; decode ~130 tok/s on prose with one request and the drafter (2×3090, capped, 2026-10-10)
Drafter	qwen3.8_27b_idiotSavant_sm_86_dflash2: DFlash2, re-tuned against this model, 5.62 accepted tokens/step on agentic coding
Needs	vLLM + mandatory online Hadamards: 512-block before `down_proj`, one per head before `o_proj` / `out_proj` (Genesis PN148 + PN154; see Serving)
Weights	2026-09-28, updated in place: per-head Hadamard on `o_proj`/`out_proj` and GDN gates `in_proj_a/b` in int4 (KL 0.0194 → 0.0178). This card: 2026-10-10

🧰 Serving stack: everything this card relies on — the vLLM patches, the integer kernels, the tree speculation and the exact compose it is measured with — lives in blairq/genesis-vllm-patches. Known-good setup: compose/docker-compose.qwen38-27b-idiotsavant-sm86.yml.

⚠️ This is not a drop-in int4 checkpoint. The weights live in a rotated basis: down_proj expects its input multiplied by a 512-block Hadamard, and o_proj / out_proj by a Hadamard of one head (256 / 128). Without them the model outputs garbage. Small hooks are enough (see below).

Benchmarks against the BF16 model card

Table with columns: Benchmark, idiotSavant, Qwen3.8-27B BF16 (card), Δ, n
Benchmark	idiotSavant	Qwen3.8-27B BF16 (card)	Δ	n
IFBench (prompt-level loose)	83.0 ±4.3	79.5	+3.5	300
GPQA Diamond	91.9 ±3.8	89.2	+2.7	198
HLE (fixed sample)	23.6 ±3.7 (28.2 closing `</think>`¹)	30.8

± is the 95% interval of the local score. ¹ Closing the </think> that 81 HLE answers never emitted; Genesis PN175 now does it at serving time (see below).

How to read it.

Short reasoning: a tie. IFBench, GPQA and ERQA sit inside their intervals: no visible loss from the quantization. With enough token budget LiveCodeBench ties too (90.8 vs 90.3).
Long reasoning and fine perception: real losses. HLE and BabyVision lose.
What goes wrong in long reasoning, measured.
- Truncation: with the first 81 920-token cap, 16% of HLE and 10 of 131 LiveCodeBench answers were cut while still thinking. With 200k only 8 of 500 HLE answers hit the cap, and LiveCodeBench went from 85.5 to 90.8. The card budgets up to 262k tokens of reasoning.
- Unclosed reasoning: 16% of HLE answers end without emitting </think>; the answer is written inside the reasoning and the parser drops it. The rate grows with length: 0% below 8k tokens, 8% at 16–32k, 25% at 64–82k.
- Grading those answers anyway gives HLE 28.2. Forcing the close instead (append </think> and let the model write the answer) gets 23 of those 81 right, against 24 by grading the reasoning. That is what Genesis PN175 does at serving time. Whether the BF16 does the same at this effort is not measured.

Not an apples-to-apples comparison.

Different reference model. The BF16 column is Qwen's own number for the official model. This one is quantized from orcarouter's abliterated BF16, so the gap mixes abliteration, quantization and harness differences.
Judge. HLE, CharXiv and BabyVision are judged by this same model, not by GPT-4o as in the card.
Smaller splits for HLE and MathVision (see n).
Not reproduced: the agentic rows of the card (SWE-bench Pro, Terminal Bench, OSWorld, …) need their own environments.

Suite and raw results: benchmarks/qwen38_card and summary.md (200k re-run).

Why "idiot savant"

Table with columns: linear, int8 per-token error, no rotation, worst layers, crest
linear	int8 per-token error, no rotation	worst layers	crest
k_proj / v_proj	10–14%	37–48% (3, 7, 11, 15)	~56
in_proj_qkv (GatedDeltaNet)	12%	24% (6)	~58
gate / up	5–7%	12%	~32
down_proj	6%	11%	~27

Results

Table with columns: torch W4A16, torch W4A8, vLLM W4A8 (as served), top-1
	torch W4A16	torch W4A8	vLLM W4A8 (as served)	top-1
noon AutoRound int4 (popular)	0.0155	0.0300	0.0385	0.938
GPTQ, not rotated	0.0122	0.0289	0.0365	0.940
idiotSavant, 2026-09-25	0.0104	0.0116	0.0194	0.956

Without rotation, int8 activations wipe out GPTQ's advantage over AutoRound. With rotation, int8 activations cost only +12% KL.

Speed on 2× RTX 3090, TP=2, power-capped (229 / 243 W), vLLM 0.29.0. Rotation against no rotation, measured back to back on 2026-09-25:

Table with columns: prefill 28k, prefill 57k, decode (1 req, no spec.)
	prefill 28k	prefill 57k	decode (1 req, no spec.)
GPTQ, not rotated	2699 tok/s	2293 tok/s	~58 tok/s
idiotSavant (fused kernel)	2721 tok/s	2298 tok/s	~60 tok/s

What was done, layer by layer

Row convention y = x·Wᵀ. R = diag(±1 signs, seed 20260924) · block-Hadamard(1024). g = 1 + w is the GemmaRMSNorm weight, folded into the next linear (the norm itself becomes w = 0).

Table with columns: where, stored weight, Hessian for GPTQ, why
where	stored weight	Hessian for GPTQ	why
q/k/v, in_proj_qkv/z, gate/up	`W·diag(g)·Rᵀ`	`R·H_n·Rᵀ`	they read the residual: that is where the spike lives
o_proj, out_proj	`R·W·Hc`	`Hc·H·Hc`	they write the residual; their input (crest 12–21) is the only one not rotated, so `Hc` = Hadamard of one head (256 attention, 128 GDN), applied online
down_proj

Quantization: GPTQ, int4 symmetric, group 128, fp16 scales = max|w|/7.5, damp 0.01, no act-order (Marlin reads contiguous groups).
Calibration: 256 × 4096 tokens of real agentic-coding traffic, tokenized with the served chat template.
Layer 7 gotcha: it has a norm channel with g = 0 exactly, on the massive-activation channel. After rotation that energy leaks into the MLP unless the Hessian is taken on the pre-norm-weight input n. Done here.
MTP head: dropped. The drafter is DFlash2, and the MTP head would be invalid in the rotated basis.

The full rationale, every alternative that was measured, and the numbers behind each decision are in reproducir/DECISIONES.md.

Serving

vLLM (tested: v0.29.0, 2× RTX 3090, TP=2)

bash

GENESIS_ENABLE_PN148_ROT_DOWN=1       # mandatoryGENESIS_ENABLE_PN154_HAD_SALIDAS=1    # mandatory (per-head Hadamard before o_proj / out_proj)GENESIS_ENABLE_PN155_BA_EN_QKVZ=1     # GDN gates in int4 inside the in_proj_qkvz MarlinGENESIS_ENABLE_PN149_ROT_BORRADOR=1   # only if you use the DFlash2 drafterVLLM_MARLIN_INPUT_DTYPE=int8          # W4A8 (without it: W4A16, also works) vllm serve BlairQ/qwen3.8_27b_idiotSavant_sm_86 \  --quantization compressed-tensors --dtype float16 --tensor-parallel-size 2 \  --kv-cache-dtype int8_per_token_head --max-model-len 262144 \  --speculative-config '{"method":"dflash","model":"BlairQ/qwen3.8_27b_idiotSavant_sm_86_dflash2","num_speculative_tokens":8}'

8-node tree speculation with the DFlash2 drafter;
int8 KV (int8_per_token_head) read by an integer PTX decode kernel, and an int8 P2P all-reduce;
prefix cache that survives multi-turn agent work: sparse GDN checkpoints plus the reuse boundary (PN168), and an L2 KV offload to RAM that actually hits on this hybrid model with speculation (PN170/PN171). On a replayed real opencode session (threads of ~100K and ~125K plus a sub-agent) 70% of prompt tokens come from cache and every follow-up turn is 93–97% cached; a 100K turn evicted from the GPU comes back from RAM with TTFT 55.9 s → 4.3 s. The L2 is 8 GiB since 2026-10-10 (was 12), and PN174 keeps a crashed engine from leaving it behind in /dev/shm;
the multimodal processor cache at 1 GiB instead of vLLM's 4 GiB, which is duplicated in the API server and the engine and filled a 26 GB container until OOM;
scheduling for agents: one prefill at a time with priority bypasses, and a dynamic prefill chunk so a turn that is generating is not slowed to the pace of another thread's prefill;
tool-calling fixes for long contexts.

Details and measurements: the repo README.

Anything else (transformers, other engines)

python

import torch def block_hadamard(x, b=512):    h = torch.ones(1, 1, dtype=torch.float64)    while h.shape[0] < b:        h = torch.cat([torch.cat([h, h], 1), torch.cat([h, -h], 1)], 0)    H = (h / b ** 0.5).to(x.device, x.dtype)    return (x.reshape(*x.shape[:-1], -1, b) @ H).reshape(x.shape) HAD = {"mlp.down_proj": 512, "self_attn.o_proj": 256, "linear_attn.out_proj": 128}   # block = one head for o/out_proj for layer in model.model.language_model.layers:          # after from_pretrained(...)    for name, b in HAD.items():        try:            mod = layer.get_submodule(name)              # attention layers have o_proj, GDN layers out_proj        except AttributeError:            continue        mod.register_forward_pre_hook(lambda m, a, b=b: (block_hadamard(a[0], b),) + a[1:])

Reproduce

bash

hf download BlairQ/qwen3.8_27b_idiotSavant_sm_86 --include "reproducir/*" --local-dir idiotsavantcd idiotsavant/reproducirbash reconstruir.sh --dry-run        # venv + downloads the BF16 (55 GB) + validates everything (~20 s)setsid nohup bash reconstruir.sh > reconstruir.log 2>&1 &              # ~85 min on 2× RTX 3090bash correr.sh idiotsavant.py estado --trabajo trabajo --salida qwen3.8_27b_idiotSavant_sm_86   # progress (--json for scripts)

Isolated: the virtualenv and every cache stay inside the folder, so nothing is installed system-wide.
Resumable: calibration and quantization run as separate processes, one per GPU, and relaunching resumes what was done.
Resource-aware: it checks RAM (including cgroup limits), GPU and disk, and waits instead of crashing.
Other calibration: calibracion.py builds your own from the public source dataset (--desde-hf) or from your own chat logs (--conversaciones).

Limitations

Uncensored. It inherits the abliteration of the base model: refusals were removed. You are responsible for how you use it.
Tuned and measured for agentic coding with tools on sm_86. It runs elsewhere, but the numbers above are only for that.
It needs the online Hadamards (see above), so it is not a drop-in int4 checkpoint.
Long reasoning degrades. Beyond ~16k tokens of thinking it sometimes ends without closing </think>, and at xhigh effort it can run past an 81 920-token budget (HLE: 12% and 16% of the answers). Give it a large max_tokens, or a lower reasoning_effort, for hard problems.
Vision. The vision tower is kept as-is in bf16, but the language model that reads its tokens is int4. It holds up on charts, documents and real-world photos (CharXiv −2.4, RealWorldQA −3.9) and loses most on fine visual discrimination (BabyVision −19.6).

Credits and references

Base model: Qwen/Qwen3.8-27B, abliterated by orcarouter.
Methods: GPTQ (Frantar et al., 2022) · QuaRot (arXiv 2404.00456) · SpinQuant (2405.16406) · MambaQuant (2501.13484) · KL-lens sensitivity for hybrid models (2604.13440).
Kernels and serving: vLLM and Marlin; the Genesis patches for vLLM (blairq/genesis-vllm-patches, forked from Sandermage/genesis-vllm-patches).
prompts derived from (CC-BY-4.0), shipped tokenized in .

Resumen en español

Contra el BF16 en los benchmarks del card de Qwen (2026-10-10, ver la tabla de arriba):

Empata en razonamiento corto: IFBench 83,0 contra 79,5; GPQA 91,9 contra 89,2; ERQA 67,5 contra 65,5.
LiveCodeBench empata con tope de 200k tokens: 90,8 contra 90,3 (con 81 920 daba 85,5).
Pierde en HLE: 23,6 contra 30,8 con tope de 200k; 28,2 si se cierra el </think> que falta.
Pierde en percepción fina: BabyVision 46,1 contra 65,7.
Lo que falla en el razonamiento largo es que el modelo termina sin cerrar </think>, y eso pasa más cuanto más largo es el razonamiento. Genesis PN175 lo cierra al servir y el modelo responde.

Todas las decisiones, con sus mediciones, están en reproducir/DECISIONES.md. Los parches, el compose con el que se sirve y la suite de benchmarks: genesis-vllm-patches.