XudongZhu

Qwen3-1.7B-Base-SFT-OffPolicy-BF16Delta

README

License: apache-2.0

Checkpoints

This repository does not hold a full copy of each checkpoint. It holds step 300 as a normal checkpoint at the root, and deltas that let you rebuild the other 74 saved steps yourself. Each file in deltas/ is the XOR of a step's raw BF16 bits with those of the step before, compressed with zstd. The chain starts from the base model at revision ea980cb, which is not stored here, and manifest.json holds the SHA-256 of every delta and every step. A rebuilt step is bit-identical to the FP32 checkpoint of Qwen3-1.7B-Base-SFT-OffPolicy rounded to BF16. The low FP32 bits are not kept: for those, use that repository. Step 0 is the base model itself.

Step 300 loads directly:

python

from transformers import AutoModelForCausalLM, AutoTokenizer repo = "XudongZhu/Qwen3-1.7B-Base-SFT-OffPolicy-BF16Delta"model = AutoModelForCausalLM.from_pretrained(repo)tokenizer = AutoTokenizer.from_pretrained(repo)

For another step, rebuild it with restore.py, which needs numpy, torch, safetensors, huggingface_hub and the zstd command. It downloads the base model and walks the chain from step 1, about 6 seconds per step on a CPU:

bash

hf download XudongZhu/Qwen3-1.7B-Base-SFT-OffPolicy-BF16Delta --local-dir chainpython chain/restore.py chain --step 100 --out step100

step100 is then a normal checkpoint folder for from_pretrained. The steps are 1 to 50, then 60, 70, …, 300. --verify rebuilds every step and checks its SHA-256.

Prompt format

The user turn is the problem, a blank line, and Please reason step by step, and put your final answer within \boxed{}. There is no system prompt; tokenizer.apply_chat_template builds this:

text

<|im_start|>user{problem} Please reason step by step, and put your final answer within \boxed{}.<|im_end|><|im_start|>assistant

Training

Data. From the default split of OpenR1-Math-220k (93,733 problems):

Keep the math word problems that have a DeepSeek-R1 answer which math-verify marks correct and which is complete: 59,188 problems. (16,115 are not word problems, and 18,430 have no such answer; none is also in MATH-500.)
Shuffle them with seed 0 and hold out the first 500 for evaluation.
In this order, skip the problems whose prompt is longer than 1,024 tokens (29) or whose prompt and R1 answer together are longer than 16,384 tokens (684), and take the first 38,400.

The run trains on these 38,400 problems in this order, once, each with its R1 answer from step 1 (one picked at random, with seed 0, when there are several).

One step. One optimizer step on the R1 answers of 128 problems, with the SFT loss: the mean of −log p over the answer tokens, without the prompt.

Table with columns: Hyperparameter, Value
Hyperparameter	Value
Optimizer	AdamW (β = 0.9, 0.999), weight decay 0, gradient clipping 1.0
Learning rate	1e-6, constant, no warmup
Batch	128 problems with their R1 answers per step, one optimizer step
Length	prompt and answer ≤ 16,384 tokens
Steps	300, one pass over the data
Precision	FP32 weights; BF16 compute and FP32 gradient reduction (FSDP mixed precision)
Seeds	0 (the R1 answer picked per problem)

Uses and limitations

For research on training dynamics: compare weights or behavior across checkpoints, and against the on-policy twin. The run trains only on English math problems, with no instruction tuning or safety training beyond the base model; its answers are long chains of reasoning.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

XudongZhu

Model Tree

Base

Qwen/Qwen3-1.7B-Base

Fine-tuned

this model

Input Modalities

Text

Output Modalities