ExploreXploitQ
DeepMath-103K-Level6-Qwen3-4B-Base-GRPO
Available on FriendliAI
Dedicated EndpointsRun this model inference on single tenant GPU with unmatched speed and reliability at scale.
Learn moreContainerRun this model inference with full control and performance in your environment.
Learn moreModel Details
Supported Functionality
Dedicated EndpointsContainer
README
License: apache-2.0
Dataset
The training data consists of 57,046 examples from the DeepMath Level 6 training file available in Keven16/G-OPD-Training-Data, originating from zwhe99/DeepMath-103K.
Training Details
Training configuration
- Base model:
Qwen/Qwen3-4B-Base
- Training framework: verl
- Algorithm: GRPO
- Parameter update: Full-parameter fine-tuning
- Rollout engine: vLLM
- Context length: 32,768 tokens
- Responses per prompt: 8
- GRPO outcome weight:
1.0
- Prompt length: 1,024 tokens
- Response length: 7,168 tokens
- Max model length: 32,768 tokens
- Rollout temperature:
1.0
- /
Dataset
- Training dataset: DeepMath-103K Level 6
- Training examples: 57,046
- Training-time validation datasets: AIME25, AMC22–23, AIME24
- Validation questions: 143
Validation Accuracy
- AIME 2025 (avg@16): 21.25%
- AMC 2022–2023 (avg@16): 60.77%
- AIME 2024 (avg@16): 23.54%
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "ExploreXploitQ/DeepMath-103K-Level6-Qwen3-4B-Base-GRPO"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype="auto",
device_map="auto",
)
Rollout top-p / top-k:
1.0
-1
Repetition penalty: 1.0KL loss: DisabledFormat reward: DisabledLearned reward model: DisabledLoss aggregation: token-meanLearning rate: 1e-6Learning-rate schedule: Constant; no warmupWeight decay: 0.01PPO mini-batch size: 64PPO micro-batch size per GPU: 1Number of GPUs: 4Number of epochs: 1Save frequency: Every 20 stepsTest frequency: Every 20 stepsValidation sampling: 16 responses per prompt; temperature 1.0; top-p 0.95Validation response length: 31,744 tokens