Results
Table with columns: Released, SFT, RL additive, RL gated | Released | SFT | RL additive | RL gated |
|---|
| Answer correct | 76.2% | 61.4% | 60.1% | 61.7% |
| Reasoning in the requested language | 39.8% | 99.4% | 99.6% | 99.7% |
| Did not finish | 4.7% | 21.1% | 1.2% | 1.8% |
| Delivered (J@1) | 28.6% | 60.2% | 59.8% | 61.4% |
| Delivered, best of 16 (J@16) | 35.7% | 84.6% | 83.1% | 83.7% |
| Median response, tokens | 5,824 | 7,673 | 4,072 | 4,463 |
70 competition problems (30 AIME-2026 + 40 AMC-2023) per language from Multilingual-Math-Reasoning-Eval, 16 samples each at temperature 0.7, 24,576-token budget. A response is delivered when the answer is correct, the visible reasoning is in the requested language, and it terminates (J = C ∧ L ∧ T, scored per response). Tables show the mean over Chinese, Spanish, French, Arabic and Russian.
Table with columns: Correct, In language, Did not finish, J@1, J@16, Median tokens | Correct | In language | Did not finish | J@1 | J@16 | Median tokens |
|---|
| English | 65.7% | 100.0% | 0.2% | 65.7% | 85.7% | 3,510 |
| Chinese | 58.7% | 99.7% | 0.6% | 58.5% | 81.4% | 3,596 |
| Spanish | 61.5% | 99.8% | 0.8% | 61.4% | 81.4% | 3,856 |
| French | 61.3% | 99.8% | 0.3% | 61.3% | 85.7% | 3,818 |
| Arabic | 57.3% | 98.6% | 2.1% | 56.1% | 82.9% | 4,566 |
| Russian | 61.7% | 99.9% | 2.0% | 61.7% | 84.3% | 4,524 |
Table with columns: Delivered (J), non-English mean, Released, SFT, RL additive, RL gated| Delivered (J), non-English mean | Released | SFT | RL additive | RL gated |
|---|
| PolyMath, 444 problems | 16.8% | 34.1% | 35.4% | 36.8% |
| MMLU-ProX-Lite, 588 questions | 24.0% | 49.0% | 56.2% | 55.5% |
Quick start
from datasets import load_datasetfrom transformers import AutoModelForCausalLM, AutoTokenizer name = "96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive"tok = AutoTokenizer.from_pretrained(name)model = AutoModelForCausalLM.from_pretrained(name, torch_dtype="auto", device_map="auto") # a French problem with a French instruction: "…en raisonnant entièrement en français."prompt = load_dataset("96kevinli29/Multilingual-Math-Reasoning-Eval", "fr")["amc23"][0]["prompt"]text = tok.apply_chat_template([{"role": "user", "content": prompt}], tokenize=False, add_generation_prompt=True)inputs = tok(text, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=24576, do_sample=True, temperature=0.7)print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
Qwen3 chat template with thinking on; the reasoning appears inside <think> … </think>. Give hard problems a large max_new_tokens. Loads with transformers 4.x and 5.x.
Table with columns: Instruction that opens the user turn | Instruction that opens the user turn |
|---|
| English | Solve the following math problem step by step, reasoning entirely in English. |
| Chinese | 请全程用中文逐步推理并解决以下数学问题。 |
| Spanish | Resuelve el siguiente problema matemático paso a paso, razonando enteramente en español. |
| French | Résous le problème de mathématiques suivant étape par étape, en raisonnant entièrement en français. |
| Arabic | حل المسألة الرياضية التالية خطوة بخطوة مع التفكير بالكامل باللغة العربية. |
| Russian | Решите следующую математическую задачу шаг за шагом, рассуждая полностью на русском языке. |
The full prompts (instruction, problem and answer format, all in the target language) are in the evaluation set.
Model family
Data: SFT-Math-Multilingual-43k · SFT-Math-Multilingual-56k · DAPO-Math-Multilingual-6Lang · Multilingual-Math-Reasoning-Eval
Training
Table | |
|---|
| Algorithm | PPO, GAE with γ = λ = 1, clip ratio 0.2, no KL term |
| Reward | 0.7 · C + 0.3 · ℓ, plus a soft penalty for overlong responses |
| Prompts | DAPO-Math-Multilingual-6Lang, 16,754 localized DAPO-Math-17K prompts |
| Batch | 32 prompts × 8 rollouts per step, 16,384-token response cap |
| Learning rate | 1e-6 actor, 1e-5 critic |
| Steps | 200 |
| Hardware · framework | 8 GPUs on one node · verl, FSDP |
C is rule-based correctness of the final answer; ℓ ∈ [0, 1] is a language-identification score of the response against the language of the prompt.
Limitations
- Accuracy stays below the released Qwen3-4B on the same problems (76.2% → 60.1%). The cost enters with SFT; RL does not recover it, and it is now what limits delivery.
- 200 PPO steps, one seed, one model size.
- Mathematics and six languages only.
Citation
@article{li2026posttraining, title = {What Does Post-Training Change in Multilingual Reasoning?}, author = {Li, Hongyang and Li, Xiao and Wu, Caesar and Danoy, Gr{\'e}goire and Bouvry, Pascal}, journal = {arXiv preprint arXiv:2609.37104}, year = {2026}} @misc{qwen3-4b-mathrl-multilingual-additive-2026, title = {{Qwen3-4B-MathRL-Multilingual-Additive}: PPO from multilingual-SFT {Qwen3-4B} with a additive language reward}, author = {Li, Hongyang and Li, Xiao and Wu, Caesar and Danoy, Gr{\'e}goire and Bouvry, Pascal and {Sea-Fill Community}}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/96kevinli29/Qwen3-4B-MathRL-Multilingual-Additive}}, note = {Maintained by Sea-Fill Community.}}
Built and released by the authors together with the Sea-Fill Community, which maintains this release.