Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-27B", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "hangyiw/Qwen3.6-27B-TRAP-QA-GRPO-LoRA")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B")
Training
Table | |
|---|
| Base model | Qwen/Qwen3.6-27B |
| Method | GRPO with a rubric-based reward |
| LoRA | rank 32, alpha 64 |
| Rollouts | 8 per question, all 294 questions |
| Effective batch | 32 sequences |
| Learning rate | 1e-5 |
| Completion cap | 8,192 tokens |
| Epochs | 6 |
| Stabilizers | clip-higher (0.2 / 0.28), dynamic sampling, KL penalty to base |
Trained with ms-swift on a single H200
MIG partition. trainer_state.json holds the full reward history.
Results
Evaluated against the base model under identical decoding — greedy, thinking at
maximum effort, 16,384 new tokens.
Table with columns: Benchmark, Domain, Base, Tuned| Benchmark | Domain | Base | Tuned |
|---|
| CommonsenseQA | Commonsense | 87.5 | 88.2 |
| GPQA-Diamond | Graduate science | 84.9 | 86.4 |
| MMLU-Pro | Broad knowledge | 85.7 | 85.9 |
| BeyondAIME | Competition math | 4.0 | 6.0 |
|
Six of the seven improve, with code reasoning unchanged; none regress.
Links
License
Apache 2.0, following the base model.