The failure
With thinking enabled (temp 0.6, top-p 0.95, top-k 20, presence penalty 1.5), the base
model loops semantically: it re-derives the same reasoning step in different words
("Wait, 30−6=24. Wait, 30−6=24. ...") with zero exact n-gram repetition, and burns
its whole token budget. On 18 unseen train problems (16 GSM8K + 2 MATH, 16,384-token
cap): only 2/18 (11%) closed their thinking. Presence penalty 1.5 vs 0 makes no
difference — the failure is in the weights.
The edit
For exactly 64 intermediate neurons per layer (the top-64 by |mean activation
difference| between looping and finished-thinking contexts), in
layers.15.mlp.down_proj and layers.19.mlp.down_proj:
W[:, j] <- W[:, j] - 6 * (W[:, j] . d_hat) * d_hat
where d is our diagnosed loop direction (mean MLP-output difference between looping
and productive contexts, mapped through down_proj and normalized; see diagnosis.json).
This over-corrects each chosen column to 6x past orthogonality with the direction — the
same form as the source paper's edit, with a direction recovered independently
(cos 0.35 / 0.45 vs. theirs; the loop subspace is evidently robust to which direction
you pick at this strength). 131,072 weight values touched (0.016% of the model);
embeddings, output head and final norm untouched.
Measured effect (18 problems, 1 sample, fp16, HF generate)
Table with columns: model, thinking closed, boxed answer, correct| model | thinking closed | boxed answer | correct |
|---|
| base | 2/18 (11%) | 2/18 | — |
| this edit | 15/18 (83%) | 14/18 | 12/18 (67%) |
| paper's edit (reproduction) | 17/18 (94%) | 14/18 | 12/18 (67%) |
13 of the 16 base-model loopers were rescued; no regressions. Small-n caveat: 18
problems, 1 sample each — this is a mechanism replication, not a benchmark score.
Diagnosis details
Layer 15: cos(ours, paper's direction) = 0.350. Layer 19: 0.449. The paper's direction
is available in their logs repo (RikiyaT/lm_understand-logs, exp8_math). Full procedure
and logs: agent-driven recreation on Kaggle T4s; the edit was applied with the paper's
own apply_edit.py conventions (bf16 rounding).
Usage
Same as the base model; thinking mode is where the difference shows:
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
Sampling: temperature 0.6, top_p 0.95, top_k 20, max 16384-32768 tokens.