PJiNH

QFFT-repro-LIMO-SFT-7B

README

License: apache-2.0

Training

Table with columns: Item, Value
Item	Value
Method	Standard SFT. Each example is the system prompt "Please reason step by step, and put your final answer within \boxed{}.", then the question, then the response. The loss is on the response (LLaMA-Factory template `qwen`).
Data	`limo_sft.json`: the 817 questions and responses of GAIR/LIMO
Framework	LLaMA-Factory (bundled in the GitHub repository), full fine-tuning, bf16
Hyperparameters	cutoff length 16,384; 6 epochs; learning rate 1e-5 with a cosine schedule and warmup ratio 0.1; batch size 1 with gradient accumulation 8 (effective 8); weight decay 1e-4; seed 42
Hardware	1 × NVIDIA RTX PRO 6000 Blackwell; DeepSpeed ZeRO-2 with CPU optimizer offload; SDPA attention
Config	`LLaMA-Factory/examples/train_qfft/train_limo_sft.yaml`
Final train loss / runtime	0.3435 / 21,738 s

The questions and responses are exactly those of GAIR/LIMO (817/817, same order). The hyperparameters above match the paper's Table 5, which states that SFT and QFFT share one set of hyperparameters.

Known differences from the original setup:

Input format (our choice): SFT is the system prompt plus the question. The paper does not specify the SFT input format, and the authors released neither SFT data nor SFT checkpoints.
Hyperparameter source: the paper's Appendix A.2 says the SFT baselines follow the official S1, LIMO and Sky-T1 hyperparameters, which contradicts Table 5. We used Table 5.
GPU count and loss grouping: we used 1 GPU with gradient accumulation 8. The paper gives the batch size (8) but not the GPU count. The authors' released QFFT checkpoints record gradient accumulation 2.
Attention and DeepSpeed: SDPA instead of FlashAttention-2, and DeepSpeed ZeRO-2 with CPU optimizer offload instead of ZeRO-3 from the upstream config.
Library versions: transformers 4.52.1 and PyTorch 2.13, newer than the authors' checkpoints (transformers 4.51.0, PyTorch 2.6).

Evaluation (this reproduction)

Settings follow the released eval/eval.sh:

Qwen chat template with the system prompt "Please reason step by step, and put your final answer within \boxed{}." The paper's Figure 8 writes $\boxed{}$ ; the released code, used here, has no $.
temperature 0.6, top-p 1.0, at most 32,000 new tokens, 16 samples per problem, one vLLM 0.11.0 engine on one GPU.

The paper text states a 32,768-token budget; the released code uses 32,000. The table columns are:

Acc: accuracy averaged over the 16 samples.
Tokens: mean generated length.
RAK: the paper's Definition 2, with Qwen2.5-7B-Instruct as the reference, averaged over the 16 runs.

Table with columns: Benchmark, Acc, Tokens, RAK, Paper (Acc / Tokens / RAK)
Benchmark	Acc	Tokens	RAK	Paper (Acc / Tokens / RAK)
GSM8K	89.1	2.1K	1.5	88.2 / 1.8K / 0.2
MATH500	79.7	6.6K	5.2	80.4 / 5.8K / 6.1
AIME25	16.7	20.6K	0.0	16.8 / 17.1K / 0.2

Usage

python
from vllm import LLM, SamplingParams

llm = LLM(model="PJiNH/QFFT-repro-LIMO-SFT-7B")
tok = llm.get_tokenizer()
messages = [
    {"role": "system", "content": "Please reason step by step, and put your final answer within \\boxed{}."},
    {"role": "user", "content": "What is the sum of the first 50 positive odd integers?"},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
out = llm.generate([prompt], SamplingParams(temperature=0.6, top_p=1.0, max_tokens=32000))
print(out[0].outputs[0].text)

License and data terms

Weights: Apache-2.0, inherited from Qwen2.5-7B-Instruct. The LICENSE file in this repository carries the original notice, Copyright 2024 Alibaba Cloud. The change is the full fine-tuning described above.
Training data: GAIR/LIMO, Apache-2.0 per its dataset card (the LIMO project README says MIT). Please cite LIMO (Ye et al., 2025, arXiv:2502.03387). According to the LIMO paper, its reasoning traces were produced with DeepSeek-R1 and other models, and the problems come from sources such as NuminaMath-CoT, MATH and AIME.
Full notices: THIRD_PARTY_NOTICES.md in the GitHub repository.

Citation

Please cite the QFFT paper (arXiv:2506.12860) and the LIMO paper (arXiv:2502.03387).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

PJiNH

Model Tree

Base

Qwen/Qwen2.5-7B-Instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities