What changed
Only these weights differ from the base model, 1,024 numbers in all:
Table with columns: Layer, Neuron, Change| Layer | Neuron | Change |
|---|
| 22 | 573 | output column mlp.down_proj.weight[:, 573] ×2 |
The exact new values are in edits/.
Results
MATH-500, held out from Beaker: 7.4% → 37.4% (all 500 problems, 4 answers each).
Side effects we measured for this edit, against the unedited model on the same questions: MMLU-Pro with thinking on (490 questions, 1 answer each) 32.9% → 36.9% (+4.1 ± 2.0), because the unedited model loops there too; IFEval prompt-level strict with thinking off 53.8% → 52.1% (−1.7 ± 1.6).
Evaluation settings: thinking on, temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5, up to 32,768 new tokens (Takehi et al.'s settings), seed 0, 4 answers per problem, graded with math-verify 0.9.0, served with vLLM 0.32.0.
Usage
Load it like the base model:
from transformers import AutoModelForImageTextToText, AutoTokenizer model = AutoModelForImageTextToText.from_pretrained("rilixai/Qwen3.5-0.8B-Beaker-math-9e523b95", torch_dtype="bfloat16")tokenizer = AutoTokenizer.from_pretrained("rilixai/Qwen3.5-0.8B-Beaker-math-9e523b95")messages = [{"role": "user", "content": "What is 12 × 7?"}]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
or serve it with vLLM: vllm serve rilixai/Qwen3.5-0.8B-Beaker-math-9e523b95.
License
Apache-2.0, as the base model. This is a modified version of
Qwen/Qwen3.5-0.8B: only the weights listed above
were changed. See LICENSE.