The edit
64 intermediate neurons per layer in layers.15.mlp.down_proj and
layers.19.mlp.down_proj (131,072 values, 0.016% of the model), each changed by a
per-neuron multiple of one shared direction — the "loop direction" — at strength 6:
W[:, j] <- W[:, j] - 6 * (W[:, j] . d_hat) * d_hat
No training, no gradients, no other tensors touched.
Independent verification (18 unseen train problems, 1 sample, fp16, HF generate)
Table with columns: model, thinking closed, correct| model | thinking closed | correct |
|---|
| base | 2/18 (11%) | — |
| this rebuild | 17/18 (94%) | 12/18 (67%) |
15 of 16 base-model loopers rescued, no regressions. Small-n: mechanism replication,
not a benchmark score. The paper reports MATH-500 7.9% -> 48.1% and GSM8K 13.8% -> 67.1%
under its full protocol.
The authors' original release:
RikiyaT/Qwen3.5-0.8B-weight-space-programming.
An independently-diagnosed variant of the same fix is
here.