conorfhayes

Qwen2.5-32B-ES-MATH

README

License: apache-2.0

Training setting

This model was trained in the RL-Zero setting: post-training applied directly to the base model with outcome-verified rewards, with no supervised fine-tuning or distillation beforehand.

Data — the MATH dataset restricted to levels 3–5 (Hendrycks et al., 2021; Liu et al., 2025)
Reward — binary, verifier-checked correctness of the final \boxed{} answer
Maximum response length during training — 8,192 tokens

Training parameters

As reported in the paper appendix (Table 2):

Table with columns: Hyperparameter, Value
Hyperparameter	Value
Perturbation scale σ	0.001
Learning rate α	0.0005 (σ / 2)
Population size	32
Train batch size	1024
Max response length	8,192
Rollout temperature	0.0
Total training steps	500

Evaluation

Following Yue et al. (2025), all benchmarks are sampled at:

Table with columns: Parameter, Value
Parameter	Value
Temperature	0.6
Top-p	0.95
Max tokens	16,384

Reproduce with the evaluation harness:

bash
python -m es_pass_k.math_pass_k \
  --model-path conorfhayes/Qwen2.5-32B-ES-MATH \
  --eval-dataset datasets/evaluation_suite/math/math500 \
  --k 256 --temp 0.6 --top-p 0.95 --max-tokens 16384 \
  --template math \
  --use-gpus 0

Usage

The model expects the Qwen2.5-Math prompt format, with the reasoning instruction in the system turn. Using a different template is out-of-distribution and will depress accuracy:

python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("conorfhayes/Qwen2.5-32B-ES-MATH", torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("conorfhayes/Qwen2.5-32B-ES-MATH")

question = "What is the smallest positive integer n such that n! is divisible by 1000?"
prompt = (
    "<|im_start|>system\nPlease reason step by step, and put your final answer "
    "within \\boxed{}.<|im_end|>\n<|im_start|>user\n"
    + question
    + "<|im_end|>\n<|im_start|>assistant\n"
)

out = model.generate(**tok(prompt, return_tensors="pt").to(model.device),
                     max_new_tokens=2048, temperature=0.6, top_p=0.95, do_sample=True)
print(tok.decode(out[0], skip_special_tokens=True))

Sampling matters: this model is intended for repeated sampling, where its broader output distribution is the advantage. Greedy decoding discards exactly the property ES optimises for.

Limitations

Trained and evaluated only on English mathematical reasoning. Behaviour on other domains, languages, or task formats is uncharacterised.
Inherits the capabilities and biases of Qwen/Qwen2.5-32B.
No safety alignment was performed. This is a research artefact for studying solution coverage under test-time scaling, not a deployment-ready assistant.
Reward was outcome-based only, so correct answers reached by faulty reasoning were rewarded.

Citation

bibtex
@article{hayes2026beyond,
  title   = {Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies},
  author  = {Hayes, Conor F. and Meyerson, Elliot and Schweighofer, Kajetan and
             Dailey, Roberto and Hodjat, Babak and Miikkulainen, Risto and Qiu, Xin},
  year    = {2026}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

conorfhayes

Model Tree

Base

Qwen/Qwen2.5-32B

Fine-tuned

this model

Input Modalities

Text

Output Modalities