emorywang

opsd-qwen3-1.7b-singlegpu

Deploy Dedicated

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

README

Training Config

Base model: Qwen/Qwen3-1.7B
Method: OPSD (fixed teacher, LoRA student)
LoRA: r=64, alpha=128, target=[q,k,v,o,gate,up,down]_proj
Trainer: TRL 0.26 GOLD trainer
Steps: 125 (effective batch=8)
Hardware: 1×H20 96GB, SDPA attention
Dataset: siyanzhao/Openthoughts_math_30k_opsd (29,434 examples)

Files

adapter_*.json/safetensors — Final LoRA adapter (step 125)
checkpoint-100/ — Full training state at step 100 (for paper comparison)

Usage

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "emorywang/opsd-qwen3-1.7b-singlegpu")
tokenizer = AutoTokenizer.from_pretrained("emorywang/opsd-qwen3-1.7b-singlegpu")