What changed
Only these weights differ from the base model, 2,048 numbers in all:
Table with columns: Layer, Neuron, Change| Layer | Neuron | Change |
|---|
| 15 | 349 | output column mlp.down_proj.weight[:, 349] ×4 |
| 22 | 573 | output column mlp.down_proj.weight[:, 573] ×1.25 |
The exact new values are in edits/.
Results
MATH-500, held out from Beaker: 6.9% → 34.8% (all 500 problems, 4 answers each).
Evaluation settings: thinking on, temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5, up to 32,768 new tokens (Takehi et al.'s settings), seed 0, 4 answers per problem, graded with math-verify 0.9.0, served with vLLM 0.32.0.
Usage
Load it like the base model:
from transformers import AutoModelForImageTextToText, AutoTokenizer model = AutoModelForImageTextToText.from_pretrained("rilixai/Qwen3.5-0.8B-Beaker-math-3c80f51b", torch_dtype="bfloat16")tokenizer = AutoTokenizer.from_pretrained("rilixai/Qwen3.5-0.8B-Beaker-math-3c80f51b")messages = [{"role": "user", "content": "What is 12 × 7?"}]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
or serve it with vLLM: vllm serve rilixai/Qwen3.5-0.8B-Beaker-math-3c80f51b.
License
Apache-2.0, as the base model. This is a modified version of
Qwen/Qwen3.5-0.8B: only the weights listed above
were changed. See LICENSE.