Use
The model answers in the format it was trained on: reasoning inside <think>...</think>, then a
final line ANSWER: <number>. Ask in the same way:
from transformers import AutoModelForCausalLM, AutoTokenizer name = "Liquor-06/Qwen3-1.7B-Base-GSM8K-distillops"tok = AutoTokenizer.from_pretrained(name)model = AutoModelForCausalLM.from_pretrained(name, dtype="auto", device_map="auto") question = "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?"suffix = "\n\nSolve step by step. Give the final answer on the last line as: ANSWER: <number>"inputs = tok.apply_chat_template([{"role": "user", "content": question + suffix}], add_generation_prompt=True, return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=512, do_sample=False)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Generation stops at <|im_end|>: it is listed with <|endoftext|> in generation_config.json, which
transformers and vLLM both read (vllm serve Liquor-06/Qwen3-1.7B-Base-GSM8K-distillops).
Training
- Teacher data: DeepSeek (
deepseek-flash, thinking off, temperature 0.7) answered the 7,473 GSM8K
training problems; an answer was kept when its final number matched the gold answer and it stayed
under 250 words, with up to 3 attempts per problem. 7,274 problems (97.3%) were kept:
Liquor-06/gsm8k-deepseek-verified.
- SFT: full fine-tuning (the base model's chat-template tokens are untrained, which LoRA cannot fix),
2 epochs, lr 1e-5, effective batch 32, completion-only loss.
- GRPO: reward 1 for a correct final number, else 0; 8 samples per prompt, DAPO loss, lr 1e-6,
no KL term, one epoch over the 3,044 training problems whose samples disagreed (96 steps).
- Weights are bf16 (training kept fp32 master weights); the bf16 weights score the same 81.65%.
Commands, configs and the full write-up: github.com/Zihao-Wu06/distillops.
Limitations
Trained and evaluated only on GSM8K-style word problems in English; it has not been tested on harder
math or other tasks and is not a general assistant. Training is not bit-for-bit repeatable on GPUs: an
earlier run of the same recipe scored 80.4%.
License
Apache-2.0, as for the base model. GSM8K is MIT-licensed; the training answers came from the DeepSeek
API (check DeepSeek's terms of use for your use case).