GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
SoFarSoGoodya
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
SoFarSoGoodya
Model Tree
deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
Input Modalities
Output Modalities
Supported Functionality
There are two adapters, applied in sequence:
| Path | Role | Size |
|---|---|---|
sft_adapter/ | Stage 1 — SFT LoRA (NuminaMath-CoT) | 78 MB |
./ (top level) | Stage 2 — DPO LoRA (math preference pairs), the final adapter | 77 MB |
The DPO adapter was trained on top of the SFT-merged model, so to reconstruct the final model you must apply SFT first, then DPO. Use the included merge.py:
bash
pip install transformers peft torch acceleratepython merge.py \--base deepseek-ai/DeepSeek-R1-Distill-Qwen-7B \--sft ./sft_adapter \--dpo ./ \--out ./DeepMath-merged
Or load adapters on the fly (see merge.py docstring / the GitHub examples/inference.py).
| Stage | Config | Outcome |
|---|---|---|
| SFT | 8×5090, native DDP, LoRA r=8/α=16 (target=all), lr 5e-5, 1 epoch, cutoff 2048, 1,410 steps | loss 0.73 → 0.39 |
| DPO | 1×5090, QLoRA 4-bit NF4 + double-quant, lr 1e-6, 3 epochs, cutoff 1024, 456 steps | loss 0.6947 (≈ ln 2 baseline), reward acc ~0.5–0.55 |
DeepSeek-R1-Distill-Qwen-7B inherits its biases/limits; tokenizer derives from Qwen.deepseek-ai/DeepSeek-R1-Distill-Qwen-7B (MIT).AI-MO/NuminaMath-CoT (Apache-2.0), cleaned 100k→90,217.Author: @SoFarSoGoodya — full attribution & contact in the GitHub repo.
DeepMath 是基于 DeepSeek-R1-Distill-Qwen-7B 的数学推理微调,用 LLaMA-Factory 走 LoRA SFT(NuminaMath)→ 合并 → LoRA DPO(数学偏好对) 两段式流水线。
本仓库是 LoRA adapter(非合并模型)。含两个 adapter,需按顺序使用:先 sft_adapter/(SFT),再顶层目录(DPO,最终)。用自带的 merge.py 一条命令重建完整模型(见上方命令)。
如实说明:这是一个学习/流水线演示作品,不是 SOTA 模型。SFT 收敛良好;DPO 完整跑通但提升有限(奖励准确率仅略高于随机),且未做独立评测。权重与代码采用 MIT;请保留对上游(DeepSeek / Qwen / NuminaMath)的署名。完整工程复盘见 GitHub 仓库。