modrill

math-think-s1-qwen3-4b

Deploy Dedicated

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

README

License: apache-2.0

Base model

Qwen/Qwen3-4B-Base

Training data

Dataset: math_think_s1 — 75,905 ShareGPT samples
Source: qihoo360/Light-R1-SFTData (stage1-76k.json)
Decontamination: MinHash + numeric normalization vs AIME/MATH eval prompts (1 sample removed)

Training recipe

Table
Item	Value
Framework	LLaMA-Factory
Method	Full SFT, DeepSpeed ZeRO-3
Template	`qwen3` (reasoning / thinking)
Cutoff	32768
Packing	true
Epochs	4
LR	1e-5 (cosine, warmup 10%)
Batch	1 × 16 grad accum × 4 GPUs = 64
Steps	944
Train loss	0.6066
Runtime	~39.3 h

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "YOUR_HF_USERNAME/math-think-s1-qwen3-4b"  # replace after upload
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "Find the sum of all positive integers n such that n^2 + 12n - 2007 is a perfect square."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)

Files

Release tarball includes: model.safetensors, config.json, generation_config.json, tokenizer files, chat_template.jinja, README.md, train_results.json, trainer_log.jsonl, MANIFEST.txt.

Model provider

modrill

Model tree

Base

Qwen/Qwen3-4B-Base

Fine-tuned

this model

Modalities

Input

Text

Output

Text