Qwen2.5-Math-1.5B-ES-MATH API & Inference Endpoint

Results

Held-out evaluation on the first 100 problems of MATH-500 (the upstream math500 split), greedy decoding, up to 3000 new tokens, graded with the upstream boxed-answer grader built on math-verify. The same 100 problems are used at every checkpoint.

Table with columns: checkpoint, accuracy, response ends with \boxed{}, median response (tokens), ran to the 3000-token cap, echoed the question
checkpoint	accuracy	response ends with `\boxed{}`	median response (tokens)	ran to the 3000-token cap	echoed the question
Qwen2.5-Math-1.5B (base)	33 %	55 %	970	40 / 100	25 / 100
ES iteration 10	51 %	78 %	621	19 / 100	11 / 100
ES iteration 20	59 %	88 %	528	10 / 100	5 / 100
ES iteration 30 (this model)	64 %	92 %	507	5 / 100	1 / 100

With n = 100 the standard error is about ±5 points, so read the trend, not the last digit. Per problem, 36 were gained, 28 kept, 5 lost and 31 are still wrong. Most of the gain is the base model's degenerate behaviour disappearing: with the Qwen math prompt it often repeats the question or never stops; after 30 iterations it almost always writes a solution and boxes an answer.

$Four-panel training summary$

Every evaluated response, with its grade, is in training/eval-output/, and the per-iteration training metrics are in training/metrics.jsonl. The other splits of the upstream math suite (AIME, AMC, Minerva, OlympiadBench) were not evaluated.

Training setting

RL-zero style: the base model, not the instruct model, trained directly against a binary outcome reward with the Qwen2.5-Math prompt format.

Data. The upstream math_lvl3to5_8k set, 8,523 MATH problems of levels 3–5, walked in order 32 problems per iteration, so the 30 iterations saw 960 distinct problems and no repeats.
Prompt. Qwen2.5-Math chat format with the system message Please reason step by step, and put your final answer within \boxed{}.
Reward. 1 if the last \boxed{} in the response matches the reference answer under math-verify, else 0. No format or length shaping.
Decoding. Greedy, at most 3000 new tokens, both for training rollouts and for evaluation.
ES update. Population of 30 perturbations per iteration, each a full-model Gaussian perturbation θ + σ·ε(seed) applied in place in bf16 and restored from the seed; rewards z-scored across the population; θ ← θ + α/N · Σ zᵢ εᵢ. Same σ, α, population size and update rule as upstream.

Training parameters

Table with columns: parameter, value
parameter	value
perturbation scale σ	0.001
learning rate α	0.0005 (σ / 2)
population size	30
problems per iteration	32
generation mini-batch	128 prompts per forward pass
max response length	3000 tokens
rollout temperature	0 (greedy)
iterations	30
precision / device	bf16 on Apple M5 Max via PyTorch MPS, single process
seed	42
wall time	about 197 h in total, 5–7.5 h per iteration (30 × 32 greedy rollouts of up to 3000 tokens, no batching across population members)

The upstream 14 B run used a population of 32, 1024 problems per iteration and 500 iterations on 8 GPUs; this run keeps the algorithm and shrinks only the budget.

Usage

Transformers

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "deepsai8/Qwen2.5-Math-1.5B-ES-MATH"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")  # transformers < 5: torch_dtype= problem = r"Convert the point $(0,3)$ in rectangular coordinates to polar coordinates. Enter your answer in the form $(r,\theta),$ where $r > 0$ and $0 \le \theta < 2 \pi.$"messages = [    {"role": "system", "content": "Please reason step by step, and put your final answer within \\boxed{}."},    {"role": "user", "content": problem},]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=3000, do_sample=False)print(tokenizer.decode(out[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

The bundled chat template reproduces the training prompt exactly. generation_config.json lists both <|endoftext|> and <|im_end|> as end-of-sequence tokens so chat-style generation stops cleanly; the base model's config only had the former.

llama.cpp

Ready-made GGUF files are in gguf/: Q8_0 (near-lossless) and Q4_K_M (small). The chat template is embedded, so llama-server applies it to /v1/chat/completions requests.

bash

# OpenAI-compatible server on http://localhost:8080llama-server -hf deepsai8/Qwen2.5-Math-1.5B-ES-MATH -hff gguf/Qwen2.5-Math-1.5B-ES-MATH-Q8_0.gguf --temp 0 # one-shot completion with the raw prompt formatllama-completion -hf deepsai8/Qwen2.5-Math-1.5B-ES-MATH -hff gguf/Qwen2.5-Math-1.5B-ES-MATH-Q8_0.gguf --temp 0 -n 3000 -no-cnv \  -p $'<|im_start|>system\nPlease reason step by step, and put your final answer within \\boxed{}.<|im_end|>\n<|im_start|>user\nWhat is the sum of the first 20 positive odd integers?<|im_end|>\n<|im_start|>assistant\n'

Quantized greedy outputs can differ slightly from the bf16 transformers model; the Q8_0 file solves the example problem above with the same derivation.

-hf downloads the file into llama.cpp's cache on first use; while the repo is private, export HF_TOKEN first.

To build your own GGUF from the safetensors (for example a different quantization):

bash

git clone https://github.com/ggml-org/llama.cpp && pip install -r llama.cpp/requirements/requirements-convert_hf_to_gguf.txthf download deepsai8/Qwen2.5-Math-1.5B-ES-MATH --local-dir Qwen2.5-Math-1.5B-ES-MATHpython llama.cpp/convert_hf_to_gguf.py Qwen2.5-Math-1.5B-ES-MATH --outfile es-math-bf16.gguf --outtype bf16llama-quantize es-math-bf16.gguf es-math-q5_k_m.gguf Q5_K_M

Ollama and LM Studio load the same GGUF files. A minimal Ollama Modelfile:

text

FROM ./Qwen2.5-Math-1.5B-ES-MATH-Q8_0.ggufPARAMETER temperature 0SYSTEM "Please reason step by step, and put your final answer within \\boxed{}."TEMPLATE """{{ if .System }}<|im_start|>system{{ .System }}<|im_end|>{{ end }}<|im_start|>user{{ .Prompt }}<|im_end|><|im_start|>assistant"""

Further training

The model is a standard Qwen2 checkpoint, so either trainer can pick it up by name.

On Apple Silicon (or CPU, or one CUDA GPU) with `mps_es`

bash

git clone -b mps-es-apple-silicon https://github.com/deepsaia/es-at-scale.gitcd es-at-scalepython3 -m venv .venv && source .venv/bin/activatepip install -r mps_es/requirements.txt          # datasets/ ships with the repository python -m mps_es.train_math \  --model-name deepsai8/Qwen2.5-Math-1.5B-ES-MATH \  --n-iterations 30 --batch-size 32 --mini-batch-size 128 --max-tokens 3000 \  --eval-splits math500 --eval-samples 100 --eval-freq 10 \  --experiment-name math-1.5b-continued python -m mps_es.watch experiments/math-1.5b-continued -f    # live progress; -v for detail

--n-iterations 0 evaluates without training. --eval-splits math500,amc,aime,minerva,olympiad_bench --eval-samples -1 runs the full suite. Starting from the Hub weights begins a fresh seed stream; to continue this run's exact perturbation sequence you would need its checkpoint.pt, which is not published (3.3 GB, redundant with the weights).

On GPUs with the upstream trainer (vLLM + Ray)

Install ES-at-Scale as its README describes, then:

bash

python es_at_scale/train.py --task math \  --model-name deepsai8/Qwen2.5-Math-1.5B-ES-MATH \  --sigma 0.001 --population-size 30 --n-iterations 500 --eval-freq 5 \  --train-dataset datasets/train/math_lvl3to5_8k --eval-dataset datasets/evaluation_suite/math/ \  --batch-size 1024 --mini-batch-size 1024 --max-tokens 3000 \  --n-vllm-engines 8 --use-gpus "0,1,2,3,4,5,6,7" \  --experiment-name es-math-1.5b-gpu

Limitations

Evaluated on a 100-problem slice of MATH-500 only; the ±5-point error bar is wide, and AIME, AMC, Minerva and OlympiadBench were not run.
Trained and evaluated with greedy decoding. Sampling behaviour was not tuned.
5 of 100 held-out responses still run to the 3000-token cap.
English competition mathematics only. The base model has no safety alignment and neither does this one.
The run saw 960 of the 8,523 training problems; this is a small-budget demonstration that the ES recipe works off-cluster, not a state-of-the-art math model.

Files

Table with columns: path, contents
path	contents
`model-.safetensors`, `config.json`, `tokenizer`, `chat_template.jinja`, `generation_config.json`	the model, bf16
`training_config.json`	ES hyperparameters of the run
`training/metrics.jsonl`, `training/run.json`	per-iteration training metrics and run settings
`training/eval-output/*.json`	every held-out response and its grade at baseline and iterations 10, 20, 30
`gguf/*.gguf`	Q8_0 and Q4_K_M quantizations for llama.cpp
`assets/*.png`	the figures above, light and dark

Citation

bibtex

@article{Qiu2026EvolutionStrategies,  title={Evolution Strategies at Scale: {LLM} Fine-Tuning Beyond Reinforcement Learning},  author={Qiu, Xin and Gan, Yulu and Hayes, Conor F. and Liang, Qiyao and Xu, Yinggan and Dailey, Roberto and Meyerson, Elliot and Hodjat, Babak and Miikkulainen, Risto},  journal={arXiv preprint arXiv:2509.24372},  year={2026},  eprint={2509.24372},  archivePrefix={arXiv},  primaryClass={cs.LG},  doi={10.48550/arXiv.2509.24372}}

checkpoint

accuracy

response ends with \boxed{}

median response (tokens)

ran to the 3000-token cap

echoed the question

Qwen2.5-Math-1.5B (base)

33 %

55 %

970

40 / 100

25 / 100

ES iteration 10

51 %

78 %

621

19 / 100

11 / 100

ES iteration 20

59 %

88 %

528

10 / 100

5 / 100

ES iteration 30 (this model)

64 %

92 %

507

5 / 100

1 / 100

parameter

value

perturbation scale σ

0.001

learning rate α

0.0005 (σ / 2)

population size

problems per iteration

generation mini-batch

128 prompts per forward pass

max response length

3000 tokens

rollout temperature

0 (greedy)

iterations

precision / device

bf16 on Apple M5 Max via PyTorch MPS, single process

seed

wall time

about 197 h in total, 5–7.5 h per iteration (30 × 32 greedy rollouts of up to 3000 tokens, no batching across population members)

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "deepsai8/Qwen2.5-Math-1.5B-ES-MATH"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto") # transformers < 5: torch_dtype= problem = r"Convert the point $(0,3)$ in rectangular coordinates to polar coordinates. Enter your answer in the form $(r,\theta),$ where $r > 0$ and $0 \le \theta < 2 \pi.$"messages = [ {"role": "system", "content": "Please reason step by step, and put your final answer within \\boxed{}."}, {"role": "user", "content": problem},]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=3000, do_sample=False)print(tokenizer.decode(out[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

# OpenAI-compatible server on http://localhost:8080llama-server -hf deepsai8/Qwen2.5-Math-1.5B-ES-MATH -hff gguf/Qwen2.5-Math-1.5B-ES-MATH-Q8_0.gguf --temp 0 # one-shot completion with the raw prompt formatllama-completion -hf deepsai8/Qwen2.5-Math-1.5B-ES-MATH -hff gguf/Qwen2.5-Math-1.5B-ES-MATH-Q8_0.gguf --temp 0 -n 3000 -no-cnv \ -p $'<|im_start|>system\nPlease reason step by step, and put your final answer within \\boxed{}.<|im_end|>\n<|im_start|>user\nWhat is the sum of the first 20 positive odd integers?<|im_end|>\n<|im_start|>assistant\n'

git clone https://github.com/ggml-org/llama.cpp && pip install -r llama.cpp/requirements/requirements-convert_hf_to_gguf.txthf download deepsai8/Qwen2.5-Math-1.5B-ES-MATH --local-dir Qwen2.5-Math-1.5B-ES-MATHpython llama.cpp/convert_hf_to_gguf.py Qwen2.5-Math-1.5B-ES-MATH --outfile es-math-bf16.gguf --outtype bf16llama-quantize es-math-bf16.gguf es-math-q5_k_m.gguf Q5_K_M

FROM ./Qwen2.5-Math-1.5B-ES-MATH-Q8_0.ggufPARAMETER temperature 0SYSTEM "Please reason step by step, and put your final answer within \\boxed{}."TEMPLATE """{{ if .System }}<|im_start|>system{{ .System }}<|im_end|>{{ end }}<|im_start|>user{{ .Prompt }}<|im_end|><|im_start|>assistant"""

git clone -b mps-es-apple-silicon https://github.com/deepsaia/es-at-scale.gitcd es-at-scalepython3 -m venv .venv && source .venv/bin/activatepip install -r mps_es/requirements.txt # datasets/ ships with the repository python -m mps_es.train_math \ --model-name deepsai8/Qwen2.5-Math-1.5B-ES-MATH \ --n-iterations 30 --batch-size 32 --mini-batch-size 128 --max-tokens 3000 \ --eval-splits math500 --eval-samples 100 --eval-freq 10 \ --experiment-name math-1.5b-continued python -m mps_es.watch experiments/math-1.5b-continued -f # live progress; -v for detail

python es_at_scale/train.py --task math \ --model-name deepsai8/Qwen2.5-Math-1.5B-ES-MATH \ --sigma 0.001 --population-size 30 --n-iterations 500 --eval-freq 5 \ --train-dataset datasets/train/math_lvl3to5_8k --eval-dataset datasets/evaluation_suite/math/ \ --batch-size 1024 --mini-batch-size 1024 --max-tokens 3000 \ --n-vllm-engines 8 --use-gpus "0,1,2,3,4,5,6,7" \ --experiment-name es-math-1.5b-gpu

path

contents

model-*.safetensors, config.json, tokenizer*, chat_template.jinja, generation_config.json

the model, bf16

training_config.json

ES hyperparameters of the run

training/metrics.jsonl, training/run.json

per-iteration training metrics and run settings

training/eval-output/*.json

every held-out response and its grade at baseline and iterations 10, 20, 30

gguf/*.gguf

Q8_0 and Q4_K_M quantizations for llama.cpp

assets/*.png

the figures above, light and dark

@article{Qiu2026EvolutionStrategies, title={Evolution Strategies at Scale: {LLM} Fine-Tuning Beyond Reinforcement Learning}, author={Qiu, Xin and Gan, Yulu and Hayes, Conor F. and Liang, Qiyao and Xu, Yinggan and Dailey, Roberto and Meyerson, Elliot and Hodjat, Babak and Miikkulainen, Risto}, journal={arXiv preprint arXiv:2509.24372}, year={2026}, eprint={2509.24372}, archivePrefix={arXiv}, primaryClass={cs.LG}, doi={10.48550/arXiv.2509.24372}}

Qwen2.5-Math-1.5B-ES-MATH

README

Results

Training setting

Training parameters

Usage

Transformers

llama.cpp

Further training

On Apple Silicon (or CPU, or one CUDA GPU) with mps_es

On GPUs with the upstream trainer (vLLM + Ray)

Limitations

Files

Citation

Explore FriendliAI today

README

Results

Training setting

Training parameters

Usage

Transformers

llama.cpp

Further training

On Apple Silicon (or CPU, or one CUDA GPU) with mps_es

On GPUs with the upstream trainer (vLLM + Ray)

Limitations

Files

Citation

On Apple Silicon (or CPU, or one CUDA GPU) with `mps_es`

On Apple Silicon (or CPU, or one CUDA GPU) with `mps_es`