twolven

Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-AWQ-MTP

README

License: apache-2.0

Provenance — read this first

HauhauCS released this model as GGUF only. There is no upstream BF16 safetensors checkpoint to quantize from, so this build was produced by reconstructing one:

markdown
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q8_K_P.gguf   (29.3 GiB, sha256 4e7735df4d1e...)
mmproj-...-Aggressive-BF16.gguf                          (vision tower, bf16)
        |
        +- dequantise + reconstruct -> BF16 HF safetensors (52 GB)
        |
        +- llm-compressor AWQ -------> W4A16 compressed-tensors (19 GB)   <- this repo

This is a quant of a Q8_K_P quant, not of the original weights. The ~8-bit source error is baked in and compounds with the 4-bit step. If HauhauCS ever publishes BF16 safetensors, a build made from those should be preferred over this one.

The vision tower is the exception: the mmproj GGUF ships bf16, and the tower is excluded from quantization, so it is bit-identical to upstream.

Why a reconstruction was needed

qwen3_5 is a hybrid architecture — 64 decoder layers, of which 48 are Qwen3_5GatedDeltaNet linear-attention and 16 are full attention (layer N is full-attention iff N % 4 == 3) — plus a 27-layer vision tower and an embedded NextN head. No off-the-shelf GGUF-to-HF converter handles it, so the mapping was written from scratch and proven total and bijective: 866 GGUF tensors <-> 866 non-vision HF tensors, plus 334 mmproj -> 333 HF vision tensors.

Three llama.cpp storage conventions had to be inverted. Each one loads and runs happily if you get it wrong, and produces fluent nonsense:

Table with columns: llama.cpp stores, HF expects, Inverse applied
llama.cpp stores	HF expects	Inverse applied
RMSNorm weight with `+1` folded in	raw weight	subtract 1.0 (in fp32 — see below)
`ssm_a` = `-exp(A_log)`	`A_log`	`log(-t)`
value heads grouped `(n_v, n_k)`	`(n_k, n_v)`	transpose within the head block

The value-head ordering is the one that hurts. It affects in_proj_qkv, in_proj_z, out_proj and conv1d on all 48 linear-attention layers. With it wrong the model loads, serves, and answers every prompt with degenerate repetition.

Dequantization is done in fp32, not bf16: bf16 eps at 1.0 is 0.0078, so a norm weight of 0.9666 rounds to 0.96875 and subtracting 1.0 yields -0.03125 against a true -0.0334 — a 6% error on every normalization weight in the model.

Verification

Reconstruction was gated on a numeric check against a different fine-tune of the same base, so it validates layout and storage convention, not weight identity:

Correlation sweep — every linear-attention tensor at +0.99998; A_log/dt_bias +0.97917, norm.weight +0.99219 (small tensors, at the Q8_0 noise floor). This is the check that catches permutations; the broken build scored 0.04–0.73 here.
Norm-offset sweep — all 19 norm families at +/-0.00000, confirming no residual +1.

Correlation cannot prove the weights are right, only that they are in the right shape and convention. The behavioural tests below are what establish the model works.

Quantization recipe

Table

Tool	`llm-compressor` (`AWQModifier` + `QuantizationModifier`)
Scheme	`W4A16_ASYM`, group size 128, `duo_scaling=True`
Format	`pack-quantized` (compressed-tensors)
Targets	`Linear`
Calibration	, 128 samples @ 1024 tokens, seed 42

Left in bf16 (ignore): lm_head, the entire 27-layer vision tower, the in_proj_a / in_proj_b delta-rule scalars on all 48 linear-attention layers, and the MTP head.

If you re-quantize this yourself: llm-compressor never sees model-mtp.safetensors, so the ignore list it emits omits the MTP module entirely. You must add re:.*mtp.* and re:mtp\..* to quantization_config.ignore by hand, or vLLM builds the drafter as W4A16, fails to find weight_packed, silently skips every MTP tensor and dies with KeyError: 'weight' in qwen3_5_mtp.py.

Validation

All measured on this build, 2xRTX 3090 (TP=2), vLLM nightly, fp8 KV cache, MTP n=3.

Table with columns: Test, Result
Test	Result
Coherence (5 checkable facts)	5/5
Multi-step arithmetic reasoning	pass
Vision — 3 shapes + colours from a synthetic image	3/3
Tool calling — non-streaming	pass
Tool calling — streaming	pass, no tag leakage
Needle retrieval @ 29,374 prompt tokens	pass (19 s)
Needle retrieval @ 117,374 prompt tokens	pass (96 s)
Needle retrieval @ 229,200 prompt tokens	pass (231 s)

Speed

Throughput on this build is workload-dependent by roughly 30%. MTP acceptance is much higher on predictable text than on varied prose, so tokens/second is a property of the prompt as much as of the model. No single figure is meaningful without the prompt that produced it.

Pinned measurement, 2026-08-21 — single-user (c=1), thinking disabled, 600-token generations, temperature 0, with the repetition_penalty: 1.05 default this card recommends:

Table with columns: prompt type, GPU limits 250W/275W, GPU limits 225W/225W
prompt type	GPU limits 250W/275W	GPU limits 225W/225W
expository ("describe how a four-stroke engine works")	95.9 t/s	93.3 t/s
creative prose ("write an original short story…")	74.3 t/s	72.5 t/s

Run-to-run spread was under ±1 t/s in every cell. Prompt content costs about 30%; the GPU power limit costs about 2%.

An earlier head-to-head against the Qwen3.8-27B-abliterated W4A16 build, taken in a single controlled session on the same box with the same harness and settings. The prompt was not recorded, so read this as a relative result only and not as absolute throughput:

Table with columns: median, best
	median	best
This build	95.6 t/s	96.3 t/s
Qwen3.8-27B-abliterated W4A16	84.0 t/s	93.0 t/s

MTP acceptance rate 64.9%, mean accepted length 2.95 of 4 — the draft head is genuinely doing work, which is where the margin over the comparison build comes from.

No standardized benchmarks (MMLU, GSM8K, etc.) were run. The numbers above are functional and performance checks, not a quality evaluation. Treat quality as unmeasured.

Recommended sampling settings — read this before you file a bug

Do not use greedy decoding (temperature: 0) with this model. It will emit the same sentence over and over until it hits your token cap and never produce a stop token. This is the classic Qwen3 + quantization degenerate-repetition mode, not a defect in the weights, and it is fully reproducible: greedy is deterministic, so the same prompt loops the same way every time.

generation_config.json in this repo now ships a repetition_penalty of 1.05, which is enough to prevent it. If your stack ignores generation_config.json, set it yourself:

Table with columns: parameter, thinking mode, notes
parameter	thinking mode	notes
`repetition_penalty`	1.05	the important one — do not set below 1.02
`temperature`	1.0 (repo default) or 0.6	never 0
`top_p`	0.95
`top_k`	20

Measured on this build, greedy worst case, 6000-token cap:

Table with columns: repetition_penalty, finish reason, sentence uniqueness, max verbatim repeats
`repetition_penalty`	finish reason	sentence uniqueness	max verbatim repeats
unset	`length` — never stopped	0.85	5
1.02	`stop`	1.00	1
1.05	`stop`	1.00

The penalty is not free — it costs about 6% throughput. Interleaved A/B/A/B on the same box, thinking disabled, 600-token generations, c=1: 68.4 / 67.1 t/s median with the penalty off versus 64.3 / 63.6 t/s at 1.05. That is the price of not looping; we think it is worth paying by default, and you can lower it to 1.02 if you would rather have the throughput.

It does not otherwise cost you anything measurable. Checked at 1.05 against penalty-off on the tasks a repetition penalty is most likely to damage — all identical:

Table with columns: check, penalty off, 1.05
check	penalty off	1.05
verbatim reproduction of repetition-heavy JSON	EXACT	EXACT
generated FizzBuzz, executed and asserted	PASS	PASS
12 near-identical repeated assignment lines	PASS	PASS
needle retrieval @ ~9k tokens	PASS	PASS
tool calling, streaming and non-streaming	PASS	PASS

Do not go much above 1.10 — that is where repeated code syntax and identifiers start being penalized.

Two traps if you try to fix this yourself

presence_penalty will not work as a server-side default in vLLM. vLLM only carries a fixed whitelist out of generation config into its default sampling params — repetition_penalty, temperature, top_k, top_p, min_p, max_new_tokens (ModelConfig.get_diff_sampling_param). A presence_penalty entry is silently ignored.
presence_penalty and frequency_penalty cannot be defaulted at all over the OpenAI API, because the OpenAI schema defaults them to 0.0 — clients always send them explicitly, so your default never applies. repetition_penalty defaults to None on the request and is filled from the server's defaults, which is why it is the only lever that survives a client setting its own .

Why your client probably will not save you

Coding agents commonly pin temperature: 0 for determinism and send no penalty at all. Two checked as of 2026-08-21: one sends no temperature when thinking is enabled but hardcodes temperature: 0 on its sub-agent and skill paths; the other sends no temperature and no penalty of any kind, ever. In both cases every request rides entirely on the server defaults.

Optional: `chat_template_medium.jinja`

This model's stock chat template defaults reasoning_effort to xhigh, which spends the whole budget thinking and returns an empty answer on a large fraction of requests. The default chat_template.jinja in this repo is unmodified upstream — we did not silently change the behaviour of a redistributed artifact.

For convenience an opt-in copy is included as chat_template_medium.jinja, byte-identical except for one line, which defaults reasoning_effort to medium instead:

diff
-{%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}
+{%- set resolved_reasoning_effort = reasoning_effort|default('medium') %}

Use it with --chat-template chat_template_medium.jinja, or keep the stock template and pin the value per-request with --default-chat-template-kwargs as shown below. Both work; the flag merges per-key, so clients sending their own chat_template_kwargs stay covered.

Serving with vLLM

bash
vllm serve /path/to/this/repo \
  --served-model-name qwen3.8-27b-hauhau-aggressive \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.95 \
  --max-num-seqs 4 \
  --max-num-batched-tokens 4096 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --enable-prefix-caching --enable-chunked-prefill \
  --compilation-config '{"cudagraph_mode": "PIECEWISE"}' \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --default-chat-template-kwargs '{"enable_thinking": true, "reasoning_effort": "medium"}'

Notes from getting this stable on 2x3090:

Pin reasoning_effort to medium. The Qwen3.8 chat template defaults to xhigh, which spends the entire budget thinking and returns an empty answer. Measured on this build over 4 hard prompts: at xhigh, 2 of 4 produced no answer at all (~5,400 words of reasoning, then finish_reason=length); at medium, 0 of 4 failed, averaging ~529 words of thinking and ~954 words of answer. The flag merges per-key, so clients sending their own chat_template_kwargs are still covered.
cudagraph_mode: PIECEWISE, not FULL — FULL replay segfaults with MTP (vllm#40756) at identical c=1 throughput.
On 3090s without working P2P, set NCCL_P2P_DISABLE=1 and --disable-custom-all-reduce.
Without it the tag is emitted as prose content and the subsequent SSE chunks go silent — it looks exactly like the model failing at tool use, but it is a parser bug (vLLM-side), not the weights. The "streaming tool calling: PASS, no tag leakage" result in the validation table above was produced with that patch applied.

Files

Table with columns: File, Size
File	Size
`model.safetensors`	18.7 GB
`model-mtp.safetensors`	849 MB (bf16 NextN draft head)

HauhauCS FastMTP is not used here — that is a llama.cpp-only draft sidecar needing his runtime patch. This repo carries the native embedded NextN head (blk.64 -> mtp.*), which the reconstruction preserves and vLLM consumes directly.

Provenance / credit

Fine-tune: HauhauCS — all model quality and the uncensored behaviour are theirs.
Base architecture: Qwen team, qwen3_5.
This repo contributes only the GGUF-to-HF reconstruction and the AWQ quantization.

As of 2026-08-18 no AWQ/W4A16 build of any HauhauCS Aggressive model appears on the Hub (searched model ids across 90 derivative repos; GGUF, NVFP4 and MLX builds exist).

Limitations

Quantized from Q8_K_P, not from original BF16 weights — see Provenance.
Verification establishes correct tensor layout and working behaviour, not weight-level fidelity to the upstream fine-tune.
No standardized quality benchmarks were run.
This is an uncensored model. It will answer requests that safety-tuned models refuse. You are responsible for how you use it.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

twolven

Model Tree

Base

HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Provenance — read this first

HauhauCS released this model as GGUF only. There is no upstream BF16 safetensors checkpoint to quantize from, so this build was produced by reconstructing one:

markdown
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q8_K_P.gguf   (29.3 GiB, sha256 4e7735df4d1e...)
mmproj-...-Aggressive-BF16.gguf                          (vision tower, bf16)
        |
        +- dequantise + reconstruct -> BF16 HF safetensors (52 GB)
        |
        +- llm-compressor AWQ -------> W4A16 compressed-tensors (19 GB)   <- this repo

The vision tower is the exception: the mmproj GGUF ships bf16, and the tower is excluded from quantization, so it is bit-identical to upstream.

Why a reconstruction was needed

Three llama.cpp storage conventions had to be inverted. Each one loads and runs happily if you get it wrong, and produces fluent nonsense:

Table with columns: llama.cpp stores, HF expects, Inverse applied
llama.cpp stores	HF expects	Inverse applied
RMSNorm weight with `+1` folded in	raw weight	subtract 1.0 (in fp32 — see below)
`ssm_a` = `-exp(A_log)`	`A_log`	`log(-t)`
value heads grouped `(n_v, n_k)`	`(n_k, n_v)`	transpose within the head block

Verification

Reconstruction was gated on a numeric check against a different fine-tune of the same base, so it validates layout and storage convention, not weight identity:

Correlation sweep — every linear-attention tensor at +0.99998; A_log/dt_bias +0.97917, norm.weight +0.99219 (small tensors, at the Q8_0 noise floor). This is the check that catches permutations; the broken build scored 0.04–0.73 here.
Norm-offset sweep — all 19 norm families at +/-0.00000, confirming no residual +1.

Correlation cannot prove the weights are right, only that they are in the right shape and convention. The behavioural tests below are what establish the model works.

Quantization recipe

Table

Tool	`llm-compressor` (`AWQModifier` + `QuantizationModifier`)
Scheme	`W4A16_ASYM`, group size 128, `duo_scaling=True`
Format	`pack-quantized` (compressed-tensors)
Targets	`Linear`
Calibration	, 128 samples @ 1024 tokens, seed 42

Left in bf16 (ignore): lm_head, the entire 27-layer vision tower, the in_proj_a / in_proj_b delta-rule scalars on all 48 linear-attention layers, and the MTP head.

If you re-quantize this yourself: llm-compressor never sees model-mtp.safetensors, so the ignore list it emits omits the MTP module entirely. You must add re:.*mtp.* and re:mtp\..* to quantization_config.ignore by hand, or vLLM builds the drafter as W4A16, fails to find weight_packed, silently skips every MTP tensor and dies with KeyError: 'weight' in qwen3_5_mtp.py.

Validation

All measured on this build, 2xRTX 3090 (TP=2), vLLM nightly, fp8 KV cache, MTP n=3.

Table with columns: Test, Result
Test	Result
Coherence (5 checkable facts)	5/5
Multi-step arithmetic reasoning	pass
Vision — 3 shapes + colours from a synthetic image	3/3
Tool calling — non-streaming	pass
Tool calling — streaming	pass, no tag leakage
Needle retrieval @ 29,374 prompt tokens	pass (19 s)
Needle retrieval @ 117,374 prompt tokens	pass (96 s)
Needle retrieval @ 229,200 prompt tokens	pass (231 s)

Speed

Pinned measurement, 2026-08-21 — single-user (c=1), thinking disabled, 600-token generations, temperature 0, with the repetition_penalty: 1.05 default this card recommends:

Table with columns: prompt type, GPU limits 250W/275W, GPU limits 225W/225W
prompt type	GPU limits 250W/275W	GPU limits 225W/225W
expository ("describe how a four-stroke engine works")	95.9 t/s	93.3 t/s
creative prose ("write an original short story…")	74.3 t/s	72.5 t/s

Run-to-run spread was under ±1 t/s in every cell. Prompt content costs about 30%; the GPU power limit costs about 2%.

Table with columns: median, best
	median	best
This build	95.6 t/s	96.3 t/s
Qwen3.8-27B-abliterated W4A16	84.0 t/s	93.0 t/s

MTP acceptance rate 64.9%, mean accepted length 2.95 of 4 — the draft head is genuinely doing work, which is where the margin over the comparison build comes from.

No standardized benchmarks (MMLU, GSM8K, etc.) were run. The numbers above are functional and performance checks, not a quality evaluation. Treat quality as unmeasured.

Recommended sampling settings — read this before you file a bug

generation_config.json in this repo now ships a repetition_penalty of 1.05, which is enough to prevent it. If your stack ignores generation_config.json, set it yourself:

Table with columns: parameter, thinking mode, notes
parameter	thinking mode	notes
`repetition_penalty`	1.05	the important one — do not set below 1.02
`temperature`	1.0 (repo default) or 0.6	never 0
`top_p`	0.95
`top_k`	20

Measured on this build, greedy worst case, 6000-token cap:

Table with columns: repetition_penalty, finish reason, sentence uniqueness, max verbatim repeats
`repetition_penalty`	finish reason	sentence uniqueness	max verbatim repeats
unset	`length` — never stopped	0.85	5
1.02	`stop`	1.00	1
1.05	`stop`	1.00

It does not otherwise cost you anything measurable. Checked at 1.05 against penalty-off on the tasks a repetition penalty is most likely to damage — all identical:

Table with columns: check, penalty off, 1.05
check	penalty off	1.05
verbatim reproduction of repetition-heavy JSON	EXACT	EXACT
generated FizzBuzz, executed and asserted	PASS	PASS
12 near-identical repeated assignment lines	PASS	PASS
needle retrieval @ ~9k tokens	PASS	PASS
tool calling, streaming and non-streaming	PASS	PASS

Do not go much above 1.10 — that is where repeated code syntax and identifiers start being penalized.

Two traps if you try to fix this yourself

presence_penalty will not work as a server-side default in vLLM. vLLM only carries a fixed whitelist out of generation config into its default sampling params — repetition_penalty, temperature, top_k, top_p, min_p, max_new_tokens (ModelConfig.get_diff_sampling_param). A presence_penalty entry is silently ignored.
presence_penalty and frequency_penalty cannot be defaulted at all over the OpenAI API, because the OpenAI schema defaults them to 0.0 — clients always send them explicitly, so your default never applies. repetition_penalty defaults to None on the request and is filled from the server's defaults, which is why it is the only lever that survives a client setting its own .

Why your client probably will not save you

Optional: `chat_template_medium.jinja`

For convenience an opt-in copy is included as chat_template_medium.jinja, byte-identical except for one line, which defaults reasoning_effort to medium instead:

diff
-{%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}
+{%- set resolved_reasoning_effort = reasoning_effort|default('medium') %}

Serving with vLLM

bash
vllm serve /path/to/this/repo \
  --served-model-name qwen3.8-27b-hauhau-aggressive \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.95 \
  --max-num-seqs 4 \
  --max-num-batched-tokens 4096 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --enable-prefix-caching --enable-chunked-prefill \
  --compilation-config '{"cudagraph_mode": "PIECEWISE"}' \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --default-chat-template-kwargs '{"enable_thinking": true, "reasoning_effort": "medium"}'

Notes from getting this stable on 2x3090:

Pin reasoning_effort to medium. The Qwen3.8 chat template defaults to xhigh, which spends the entire budget thinking and returns an empty answer. Measured on this build over 4 hard prompts: at xhigh, 2 of 4 produced no answer at all (~5,400 words of reasoning, then finish_reason=length); at medium, 0 of 4 failed, averaging ~529 words of thinking and ~954 words of answer. The flag merges per-key, so clients sending their own chat_template_kwargs are still covered.
cudagraph_mode: PIECEWISE, not FULL — FULL replay segfaults with MTP (vllm#40756) at identical c=1 throughput.
On 3090s without working P2P, set NCCL_P2P_DISABLE=1 and --disable-custom-all-reduce.
Without it the tag is emitted as prose content and the subsequent SSE chunks go silent — it looks exactly like the model failing at tool use, but it is a parser bug (vLLM-side), not the weights. The "streaming tool calling: PASS, no tag leakage" result in the validation table above was produced with that patch applied.

Files

Table with columns: File, Size
File	Size
`model.safetensors`	18.7 GB
`model-mtp.safetensors`	849 MB (bf16 NextN draft head)

Provenance / credit

Fine-tune: HauhauCS — all model quality and the uncensored behaviour are theirs.
Base architecture: Qwen team, qwen3_5.
This repo contributes only the GGUF-to-HF reconstruction and the AWQ quantization.

As of 2026-08-18 no AWQ/W4A16 build of any HauhauCS Aggressive model appears on the Hub (searched model ids across 90 derivative repos; GGUF, NVFP4 and MLX builds exist).

Limitations

Quantized from Q8_K_P, not from original BF16 weights — see Provenance.
Verification establishes correct tensor layout and working behaviour, not weight-level fidelity to the upstream fine-tune.
No standardized quality benchmarks were run.
This is an uncensored model. It will answer requests that safety-tuned models refuse. You are responsible for how you use it.

Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-AWQ-MTP

README

Provenance — read this first

Why a reconstruction was needed

Verification

Quantization recipe

Validation

Speed

Recommended sampling settings — read this before you file a bug

Two traps if you try to fix this yourself

Why your client probably will not save you

Optional: chat_template_medium.jinja

Serving with vLLM

Files

Provenance / credit

Limitations

Explore FriendliAI today

README

Provenance — read this first

Why a reconstruction was needed

Verification

Quantization recipe

Validation

Speed

Recommended sampling settings — read this before you file a bug

Two traps if you try to fix this yourself

Why your client probably will not save you

Optional: chat_template_medium.jinja

Serving with vLLM

Files

Provenance / credit

Limitations

Optional: `chat_template_medium.jinja`

Optional: `chat_template_medium.jinja`