What changed
Only these weights differ from the base model, 2,047 numbers in all:
Table with columns: Layer, Neuron, Change| Layer | Neuron | Change |
|---|
| 23 | 970 | output column mlp.down_proj.weight[:, 970] rewritten (the change is 2.0× the original column's length) |
| 23 | 1004 | output column mlp.down_proj.weight[:, 1004] rewritten (the change is 9.9× the original column's length) |
The exact new values are in edits/.
Results
MATH-500, held out from Beaker: 7.6% → 31.1% (all 500 problems, 4 answers each).
Evaluation settings: thinking on, temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5, up to 32,768 new tokens (Takehi et al.'s settings), seed 0, 4 answers per problem, graded with math-verify 0.9.0, served with vLLM 0.32.0.
Usage
Load it like the base model:
from transformers import AutoModelForImageTextToText, AutoTokenizer model = AutoModelForImageTextToText.from_pretrained("rilixai/Qwen3.5-0.8B-Beaker-math-6760131e", torch_dtype="bfloat16")tokenizer = AutoTokenizer.from_pretrained("rilixai/Qwen3.5-0.8B-Beaker-math-6760131e")messages = [{"role": "user", "content": "What is 12 × 7?"}]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
or serve it with vLLM: vllm serve rilixai/Qwen3.5-0.8B-Beaker-math-6760131e.
License
Apache-2.0, as the base model. This is a modified version of
Qwen/Qwen3.5-0.8B: only the weights listed above
were changed. See LICENSE.