Training
- Complete reasoning traces of at most 8,000 tokens.
- 3,000 steps × 524,288 tokens/step = 1,572,864,000 training tokens.
- Sequence length 16,384; training seed 17; loop specification
12:19:3.
- Base revision:
ea980cb0a6c2ae4b936e82123acc929f1cec04c1.
- Training source revision:
e151808e0ba853ebd31d82321bfb6febc4d93d86.
Use
The evaluated environment used Transformers 4.57.6 and vLLM 0.13.0. Use a CUDA-enabled PyTorch build compatible with your NVIDIA driver.
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer path = "tyzhu/open-loopify-Qwen3-1.7B"tok = AutoTokenizer.from_pretrained(path)model = AutoModelForCausalLM.from_pretrained(path, dtype=torch.bfloat16).cuda() question = "What is the sum of all positive divisors of 36?"prompt = question + "\n" + r"Please reason step by step, and put your final answer within \boxed{}."inputs = tok.apply_chat_template( [{"role": "user", "content": prompt}], add_generation_prompt=True, return_dict=True, return_tensors="pt",).to(model.device)out = model.generate( **inputs, max_new_tokens=16384, do_sample=True, temperature=0.6, top_p=0.95, eos_token_id=[151645, 151643],)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
The supplied generation configuration includes both EOS IDs, 151645 and 151643.
Final-checkpoint evaluation
The released weights and tokenizer/configuration files match the evaluated 3000-step artifacts by SHA-256. Each checkpoint used 758 unique questions and 1,178 sampled answers, a 16,384-token output limit, temperature 0.6, and top-p 0.95. AIME reports average accuracy over eight samples per question, not pass@8; MATH-500 and GPQA-Diamond use one sample per question.
Table with columns: Benchmark, Accuracy, Output-limit truncation| Benchmark | Accuracy | Output-limit truncation |
|---|
| AIME 2024 (30 questions × 8 samples) | 10.42% | 57.50% |
| AIME 2025 (30 questions × 8 samples) | 10.83% | 50.83% |
| MATH-500 (500 questions) | 70.00% | 14.80% |
| GPQA-Diamond (198 questions) | 16.67% | 47.98% |
This is one training seed with no matched 1.7B dense-training control. These results do not isolate the effect of looping from continued training, and the 4B dense model is not a matched control. The final checkpoint is not the best checkpoint on every benchmark. Long reasoning often reaches the output limit without finishing; generation length and answer correctness should be assessed separately. No broader safety or general-purpose quality evaluation is claimed. Different batch schedules may change the exact sampled text.
Benchmark item digest: e21c6140501b7deba4e6de58d1843ec1ac9a0f7cfb1d102aac791e33fc6c723b.
Model file SHA-256: 924b779dc8186e066ef34221fd34cd1e4594d1a07f1fcc119a2c4a243c9ba715.
License
Apache-2.0, inherited from Qwen3-1.7B-Base. The base model's license is included in this repository.