k3dani

Qwen3.6-35B-A3B-FP8-cjk-damped-S07

README

License: apache-2.0

1. What was changed, exactly

Table

Tensor	`lm_head.weight` (in shard `outside.safetensors`)
Rows changed	55 424 of 248 077 — every vocabulary entry whose decoded text contains a CJK ideograph or CJK punctuation (the "raw" mask; definition taken unchanged from dnotitia/smoothie-qwen)
Operation	`W[i] := 0.7 · W[i]` for each targeted row `i` — multiplicative scaling, S = 0.7
Everything else	byte-identical to `Qwen/Qwen3.6-35B-A3B-FP8` (54 of 55 files are hard copies of the base; `outside.safetensors` differs only in the targeted rows)
Quantisation	untouched — the FP8 weights were never de-quantised; the patch was applied to the single BF16 tensor in place

The dose S = 0.7 was chosen by measurement, not by hand: it is the smallest damping in the series below that (a) yields exactly zero CJK probability mass on the production sampling path at t = 0.6, 0.8 and 1.0, and (b) leaves the Chinese probe at 36/36. It sits directly at the edge of the cost cliff (S = 0.6 already loses 45 % of the Chinese probe) and one step inside the benefit edge (S = 0.9 already leaks at t = 0.8).

2. How to use it

Exactly like the base model, with the same chat template and tokenizer. The measurements were made with vLLM 0.19.1rc1 on a DGX Spark (GB10), e.g.:

bash

vllm serve <this-repo> --served-model-name qwen36 \  --kv-cache-dtype fp8_e4m3 --enable-prefix-caching --reasoning-parser qwen3 \  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

Read this before relying on it. The damping is effective only together with a truncating sampler (top_k, top_p, or greedy). Multiplying a negative logit by 0.7 moves it towards zero, i.e. raises that token's probability — and 96.4 % of the targeted rows have negative logits on Hungarian text. In the raw (untruncated) distribution at t = 1.0 this checkpoint has more CJK mass than the base (13.4 % vs 10.5 % summed over the targeted rows, measured on the mean Hungarian hidden state). That mass sits far below the top-20 and is removed by top_k = 20; it is not removed by pure temperature sampling with no truncation. If your sampler has no top_k/top_p, do not expect this checkpoint to help.

3. Results — the whole dose–response series, not just the winner

Twelve arms were measured on the same six instruments; every arm that ran is in the table (the study's rule: no arm is dropped after the fact). K0 is the unmodified base.

Chinese probe: 36 short Chinese tasks (idioms, classifiers, antonyms, simplified/traditional conversion, classification, common knowledge), greedy, exact-match; "no Han" = share of answers that contain no CJK character at all (the model cannot even speak Chinese). Han risk = expected number of CJK tokens per million generated answer positions on Hungarian agent traces, computed from the top-20 log-probabilities at every position under the production sampler (presence_penalty 1.5, top_k 20, top_p 0.95, repetition_penalty 1.05), 95 % bootstrap CI over answers; "exact 0" means every position had zero CJK mass inside the truncated support.

Table with columns: arm, form, mask, param, Chinese probe, no Han, Han risk t=0.6, t=0.8, t=1.0, HU trap corpus (150)
arm	form	mask	param	Chinese probe	no Han	Han risk `t=0.6`	`t=0.8`	`t=1.0`	HU trap corpus (150)
`K0` (base)	—	—	—	36/36

"direction" = the row-replacement form of ThakiCloud/Qwen3.8-27B-ko-cjk-suppressed (W[i] := −α · μ_h / ‖μ_h‖², with μ_h the mean final hidden state on Hungarian text). "refined" = the Korean-style mask that keeps single-character Han tokens and only-simplified characters (54 939 rows, 3 964 frequent single ideographs left untouched).

What the table says.

The cost is entirely in Chinese, and it is a cliff, not a slope: 100 % at S ≥ 0.7, 55.6 % at 0.6, 2.8 % at 0.5. The direction form is not cheaper (α = 0.5 already kills Chinese), and the refined mask fails on both axes (it neither keeps Chinese usable nor removes the risk).
The Hungarian cost is zero by construction, not by luck. The intervention only changes logits of CJK rows; those are never the argmax on Hungarian text, so the greedy Hungarian output of an arm is byte-identical to a base served in the same instance mode on all 150 trap-corpus items. This was measured for eight arms — including S07 and S = 0.1; S08/S06 and A050 landed in instance modes with no base run, so there it is derived. Under sampling the output can differ only at positions where a CJK token was inside the truncated support, i.e. exactly the positions that were the risk. The 150-item column therefore measures the server instance, not the patch (see §5).
Confirmation on the actual sampling support. The table's risk numbers are computed from raw top-20 log-probabilities and are lower bounds (penalties are applied before top-k, so a raw rank-21+ token can enter the real support). Re-measured with the server returning post-penalty, post-truncation log-probabilities (
text
--logprobs-mode processed_logprobs
), the base comes out about an order of magnitude worse than the table — a ratio of point estimates from three traces, not a measured factor — (t=0.6: 73/M [15; 148]; t=0.3: 392/M with 4 real CJK tokens in 7 869 answer positions), while stays at at and ; on the production retry path (, , ) is — not exactly zero, with no observed event. This one is itself a lower bound: with , ties at the top-k boundary keep more than 20 candidates at ~4 % of positions.

4. What was not measured

Other models. One checkpoint, one quantisation (FP8), one engine (vLLM). Nothing here transfers to another model without re-measuring; the mask depends on the vocabulary.
Japanese and Korean. The Chinese probe is the only CJK capability instrument. Kanji and hanja share the targeted Unicode ranges, so the loss is expected to be the same — but it was not measured.
Other languages than Hungarian on the benefit side. The mechanism is language-agnostic (it targets a token class, not a language), the evidence is Hungarian.
Long context beyond ~15 k tokens. The Hungarian agent traces are 13–19 k tokens; the synthetic documents are shorter.
Untruncated samplers (see §2).
The width of the window. S = 0.7 is the smallest dose measured with all-zero risk and intact Chinese; S = 0.8 was zero on the production path (t ≤ 0.8) but was not re-measured on the actual support. Nothing between 0.6 and 0.7 was measured.

5. Measurement caveats you should know before quoting numbers

Server-instance non-determinism. On this hardware/engine, greedy output is bit-stable within one server process but differs between freshly started processes on knife-edge items (2–5 of 150, in at least five discrete "modes" over 16 starts). Because the patch provably does not change Hungarian greedy output, any difference in the 150-item column identifies the instance mode, not the arm. VLLM_BATCH_INVARIANT=1 with --attention-backend TRITON_ATTN was verified to be reproducible across instances and batch sizes (~12 % slower); the published numbers were not measured in that mode.
The internal extraction benchmarks (customer invoices and contracts) showed no regression on any arm; they are confirmatory only and cannot be published.
Three customer traces are the source of the Hungarian agent-trace risk numbers; they are not public. Arms served in the same instance mode follow identical sampled trajectories on them (shared server RNG stream), so the zeros of the scale arms in the table are not independent samples, and a bootstrap CI of [0; 0] is a structural statement about the visited positions, not a statistical bound — the statistical bound is the Poisson one above. The 150 synthetic documents and the Chinese probe are.

6. Reproduce it

reproduction/ contains everything needed to rebuild this checkpoint from the base and to re-run the two publishable instruments:

Table with columns: file, what
file	what
`patch_lm_head.py`	the patch: `--forma szorzas --skala 0.7 --maszk nyers`; writes `KAR.json` with sign diagnostics
`celtokenek.py`, `unihan_egyszerusitett.py`, `csak-egyszerusitett.json`	target-token masks (raw and refined) from the tokenizer and Unihan 18.0.0
`mu_h_becslo.py`, `mu-h-probahalmaz-A.json`, `mu-h-probahalmaz-B.json`, `mu-h-A.json`

The 150-item Hungarian trap corpus, its harness and the 36-item Chinese probe live in the public measurement repository (link below), together with the per-arm outputs on the synthetic corpus.

Rebuild and verify:

bash

python3 reproduction/celtokenek.py --tokenizer <base> --out celtokenek.jsonpython3 reproduction/patch_lm_head.py --snapshot <base> --celtokenek celtokenek.json \    --out ./cjk-damped-S07 --kar S07 --forma szorzas --skala 0.7 --maszk nyerspython3 reproduction/ellenoriz_kar.py --alap <base> --kar ./cjk-damped-S07 --celtokenek celtokenek.jsonsha256sum ./cjk-damped-S07/outside.safetensors   # 3cbfe634c523cc2e622679847bdc78412073dc38b738a661451995f2c0a0bfc5

7. Attribution and licence

Base model: Qwen/Qwen3.6-35B-A3B-FP8, Apache-2.0. This checkpoint is released under the same Apache-2.0 licence; the modification is the one described in §1 and nothing else.
Target-class definition: dnotitia/smoothie-qwen (the Unicode-range mask, used unchanged as the "raw" mask).
Closest prior work and the direction-based form + language-specific mask that the A* and *F arms test: ThakiCloud/Qwen3.8-27B-ko-cjk-suppressed. Their model card's warning applies here in reverse: do not reuse this mask on another language as-is — what Korean must keep, Hungarian can drop, and vice versa.
A naming note for search: "smooth" on the Hub usually means SmoothQuant, a quantisation technique. This is unrelated; hence cjk-damped.

8. Study and measurement package

Study (Hungarian, pre-registered protocol, all hypotheses and negative results): https://docai.hu/kutatas/cjk-csillapitas
Public measurement package (code, synthetic corpus, per-arm outputs, result sheet): https://github.com/k3net/docai-evals/tree/master/experiments/2026-09-18-cjk-damping-dose-response-gb10

bibtex

@misc{kiss2026cjkdamped,  title  = {How much may be taken from a token class? A dose--response study of CJK damping in the output head of Qwen3.6-35B-A3B-FP8},  author = {Kiss, D{\'a}niel},  year   = {2026},  note   = {K3Net Kft. / DocAI. Model: Qwen3.6-35B-A3B-FP8-cjk-damped-S07. Pre-registered protocol 2026-09-15, measurements 2026-09-18--20.},  url    = {https://docai.hu/kutatas/cjk-csillapitas}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

k3dani

Model Tree

Base

Qwen/Qwen3.6-35B-A3B-FP8

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

1. What was changed, exactly

Table

Tensor	`lm_head.weight` (in shard `outside.safetensors`)
Rows changed	55 424 of 248 077 — every vocabulary entry whose decoded text contains a CJK ideograph or CJK punctuation (the "raw" mask; definition taken unchanged from dnotitia/smoothie-qwen)
Operation	`W[i] := 0.7 · W[i]` for each targeted row `i` — multiplicative scaling, S = 0.7
Everything else	byte-identical to `Qwen/Qwen3.6-35B-A3B-FP8` (54 of 55 files are hard copies of the base; `outside.safetensors` differs only in the targeted rows)
Quantisation	untouched — the FP8 weights were never de-quantised; the patch was applied to the single BF16 tensor in place

2. How to use it

Exactly like the base model, with the same chat template and tokenizer. The measurements were made with vLLM 0.19.1rc1 on a DGX Spark (GB10), e.g.:

bash

vllm serve <this-repo> --served-model-name qwen36 \  --kv-cache-dtype fp8_e4m3 --enable-prefix-caching --reasoning-parser qwen3 \  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

3. Results — the whole dose–response series, not just the winner

Twelve arms were measured on the same six instruments; every arm that ran is in the table (the study's rule: no arm is dropped after the fact). K0 is the unmodified base.

Table with columns: arm, form, mask, param, Chinese probe, no Han, Han risk t=0.6, t=0.8, t=1.0, HU trap corpus (150)
arm	form	mask	param	Chinese probe	no Han	Han risk `t=0.6`	`t=0.8`	`t=1.0`	HU trap corpus (150)
`K0` (base)	—	—	—	36/36

What the table says.

The cost is entirely in Chinese, and it is a cliff, not a slope: 100 % at S ≥ 0.7, 55.6 % at 0.6, 2.8 % at 0.5. The direction form is not cheaper (α = 0.5 already kills Chinese), and the refined mask fails on both axes (it neither keeps Chinese usable nor removes the risk).
The Hungarian cost is zero by construction, not by luck. The intervention only changes logits of CJK rows; those are never the argmax on Hungarian text, so the greedy Hungarian output of an arm is byte-identical to a base served in the same instance mode on all 150 trap-corpus items. This was measured for eight arms — including S07 and S = 0.1; S08/S06 and A050 landed in instance modes with no base run, so there it is derived. Under sampling the output can differ only at positions where a CJK token was inside the truncated support, i.e. exactly the positions that were the risk. The 150-item column therefore measures the server instance, not the patch (see §5).
Confirmation on the actual sampling support. The table's risk numbers are computed from raw top-20 log-probabilities and are lower bounds (penalties are applied before top-k, so a raw rank-21+ token can enter the real support). Re-measured with the server returning post-penalty, post-truncation log-probabilities (
text
--logprobs-mode processed_logprobs
), the base comes out about an order of magnitude worse than the table — a ratio of point estimates from three traces, not a measured factor — (t=0.6: 73/M [15; 148]; t=0.3: 392/M with 4 real CJK tokens in 7 869 answer positions), while stays at at and ; on the production retry path (, , ) is — not exactly zero, with no observed event. This one is itself a lower bound: with , ties at the top-k boundary keep more than 20 candidates at ~4 % of positions.

4. What was not measured

Other models. One checkpoint, one quantisation (FP8), one engine (vLLM). Nothing here transfers to another model without re-measuring; the mask depends on the vocabulary.
Japanese and Korean. The Chinese probe is the only CJK capability instrument. Kanji and hanja share the targeted Unicode ranges, so the loss is expected to be the same — but it was not measured.
Other languages than Hungarian on the benefit side. The mechanism is language-agnostic (it targets a token class, not a language), the evidence is Hungarian.
Long context beyond ~15 k tokens. The Hungarian agent traces are 13–19 k tokens; the synthetic documents are shorter.
Untruncated samplers (see §2).
The width of the window. S = 0.7 is the smallest dose measured with all-zero risk and intact Chinese; S = 0.8 was zero on the production path (t ≤ 0.8) but was not re-measured on the actual support. Nothing between 0.6 and 0.7 was measured.

5. Measurement caveats you should know before quoting numbers

Server-instance non-determinism. On this hardware/engine, greedy output is bit-stable within one server process but differs between freshly started processes on knife-edge items (2–5 of 150, in at least five discrete "modes" over 16 starts). Because the patch provably does not change Hungarian greedy output, any difference in the 150-item column identifies the instance mode, not the arm. VLLM_BATCH_INVARIANT=1 with --attention-backend TRITON_ATTN was verified to be reproducible across instances and batch sizes (~12 % slower); the published numbers were not measured in that mode.
The internal extraction benchmarks (customer invoices and contracts) showed no regression on any arm; they are confirmatory only and cannot be published.
Three customer traces are the source of the Hungarian agent-trace risk numbers; they are not public. Arms served in the same instance mode follow identical sampled trajectories on them (shared server RNG stream), so the zeros of the scale arms in the table are not independent samples, and a bootstrap CI of [0; 0] is a structural statement about the visited positions, not a statistical bound — the statistical bound is the Poisson one above. The 150 synthetic documents and the Chinese probe are.

6. Reproduce it

reproduction/ contains everything needed to rebuild this checkpoint from the base and to re-run the two publishable instruments:

Table with columns: file, what
file	what
`patch_lm_head.py`	the patch: `--forma szorzas --skala 0.7 --maszk nyers`; writes `KAR.json` with sign diagnostics
`celtokenek.py`, `unihan_egyszerusitett.py`, `csak-egyszerusitett.json`	target-token masks (raw and refined) from the tokenizer and Unihan 18.0.0
`mu_h_becslo.py`, `mu-h-probahalmaz-A.json`, `mu-h-probahalmaz-B.json`, `mu-h-A.json`

The 150-item Hungarian trap corpus, its harness and the 36-item Chinese probe live in the public measurement repository (link below), together with the per-arm outputs on the synthetic corpus.

Rebuild and verify:

bash

python3 reproduction/celtokenek.py --tokenizer <base> --out celtokenek.jsonpython3 reproduction/patch_lm_head.py --snapshot <base> --celtokenek celtokenek.json \    --out ./cjk-damped-S07 --kar S07 --forma szorzas --skala 0.7 --maszk nyerspython3 reproduction/ellenoriz_kar.py --alap <base> --kar ./cjk-damped-S07 --celtokenek celtokenek.jsonsha256sum ./cjk-damped-S07/outside.safetensors   # 3cbfe634c523cc2e622679847bdc78412073dc38b738a661451995f2c0a0bfc5

7. Attribution and licence

Base model: Qwen/Qwen3.6-35B-A3B-FP8, Apache-2.0. This checkpoint is released under the same Apache-2.0 licence; the modification is the one described in §1 and nothing else.
Target-class definition: dnotitia/smoothie-qwen (the Unicode-range mask, used unchanged as the "raw" mask).
Closest prior work and the direction-based form + language-specific mask that the A* and *F arms test: ThakiCloud/Qwen3.8-27B-ko-cjk-suppressed. Their model card's warning applies here in reverse: do not reuse this mask on another language as-is — what Korean must keep, Hungarian can drop, and vice versa.
A naming note for search: "smooth" on the Hub usually means SmoothQuant, a quantisation technique. This is unrelated; hence cjk-damped.

8. Study and measurement package

Study (Hungarian, pre-registered protocol, all hypotheses and negative results): https://docai.hu/kutatas/cjk-csillapitas
Public measurement package (code, synthetic corpus, per-arm outputs, result sheet): https://github.com/k3net/docai-evals/tree/master/experiments/2026-09-18-cjk-damping-dose-response-gb10

bibtex

@misc{kiss2026cjkdamped,  title  = {How much may be taken from a token class? A dose--response study of CJK damping in the output head of Qwen3.6-35B-A3B-FP8},  author = {Kiss, D{\'a}niel},  year   = {2026},  note   = {K3Net Kft. / DocAI. Model: Qwen3.6-35B-A3B-FP8-cjk-damped-S07. Pre-registered protocol 2026-09-15, measurements 2026-09-18--20.},  url    = {https://docai.hu/kutatas/cjk-csillapitas}}