ziliangpeng

Qwen3-4B-SmolTalk2-SFT

README

License: apache-2.0

Model description

Base: Qwen3-4B-Base (Apache-2.0)
Tuning: full-parameter supervised fine-tuning, bf16, TRL SFTTrainer
Chat style: non-thinking (no <think> reasoning); answers directly
Voice: the training data is dominated by Magpie-pipeline responses generated by Llama-3.1-405B-Instruct (~47% of samples), so the model speaks with a Llama-flavored prose style rather than the official Qwen3 instruct style — a direct consequence of SFT being behavior cloning of its teachers

Training data (1,347,981 samples, ~1.8B tokens over 2 epochs)

SmolTalk2 SFT config, 11 no-think subsets:

Table with columns: Subset, Samples, Teacher / source
Subset	Samples	Teacher / source
smol-magpie-ultra	369K	Llama-3.1-405B-Instruct (Magpie pipeline)
OpenHermes-2.5	385K	GPT-4 era (Teknium)
multilingual-8-languages	254K	Qwen (translation of magpie/constraints prompts)
smol-summarize	96K	HF pipeline
Mixture-of-Thoughts science	86K	R1-distilled, reasoning traces removed
smol-rewrite	53K	HF pipeline
systemchats-30k	34K	HF pipeline

Excluded: all think-mode subsets, function-calling/tool traces, and >2048-token samples.

Training details

Table with columns: Parameter, Value
Parameter	Value
Epochs	2
Optimizer	AdamW, lr 1e-5, cosine, 3% warmup
Global batch	256 samples (~171K tokens/step), 10,532 steps
Loss	assistant-only (prompt tokens masked out, including system/user turns)
Max sequence length	2048
Hardware	8×H100 80GB, single node, DDP, ~16.5 h
Total tokens seen	~1.8B

Recipe notes (the parts that actually mattered)

Special-token embedding repair. Qwen3-4B-Base ships with 292 special tokens whose embeddings are untrained (and pairwise identical, cosine similarity 1.0). SFT does not fix this by itself. We re-initialized those rows before training; without this, turn-ending and template tokens never get meaningful gradients and the model never learns to stop.
Assistant-only loss with template markers. TRL's assistant_only_loss with a template containing {% generation %} markers; the plain messages format with completion_only_loss silently trains on the full sequence.
Turn-end supervision. The chat template places <|im_end|> inside the assistant generation block, so the model gets direct supervision on when to stop. Verified post-training: mean P(next token = <|im_end|>) at turn ends = 0.92 (measured on held-out training samples).
Empty-think scaffold. The template inserts think / /think scaffolding as part of the assistant turn (matching Qwen3's non-thinking convention), so the model does not produce reasoning blocks. The bundled chat_template.jinja injects the scaffold into the prompt automatically — responses come out clean with no extra configuration.

Generation config

generation_config.json ships with eos_token_id = [151645, 151643] (<|im_end|>, <|endoftext|>) pre-set. Recommended sampling: temperature 0.7, top_p 0.8, top_k 20.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("ziliangpeng/Qwen3-4B-SmolTalk2-SFT", torch_dtype="bfloat16", device_map="auto")tok = AutoTokenizer.from_pretrained("ziliangpeng/Qwen3-4B-SmolTalk2-SFT") msgs = [{"role": "user", "content": "Explain quicksort in one sentence."}]ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=256, temperature=0.7, top_p=0.8, top_k=20, do_sample=True)print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

Works out of the box with vLLM (OpenAI-compatible server) and any client that uses the bundled chat template.

Evaluation

Judge: GLM-5.3 (open weights, served locally), three dimensions on a 0-3 scale, plus order-swapped pairwise comparisons (each pair judged twice with positions swapped; a win requires winning both orders; everything else counts as a tie). 990 prompts × 3 seeds, temperature 0.7.

Absolute scores (higher is better)

Table with columns: Model, coherence, helpfulness, instruction-following
Model	coherence	helpfulness	instruction-following
Qwen3-4B-SmolTalk2-SFT (this)	2.849	2.700	2.763
Qwen3-4B (official)	2.693	2.448	2.561
Qwen3-8B (official)	2.720	2.499	2.573

Pairwise decisive win rate (vs same-size official instruct)

Table with columns: Matchup, Decisive pairs, Win rate
Matchup	Decisive pairs	Win rate
this vs Qwen3-4B (official)	1860	52.2% (official 47.8%)
this vs Qwen3-1.7B (official)	1877	76.2%
Qwen3-4B (official) vs Qwen3-1.7B	—	78.2% (reference)

Reading: this model wins the head-to-head against the official same-size Qwen3-4B (52.2% of decisive pairs) and beats every official model (including 8B) on absolute rubric scores, using nothing but open data and a documented SFT recipe — no preference optimization of any kind. An earlier judging round scored this model 45.3% because its responses carried a literal empty-think scaffold prefix (the template artifact described above, ~7pp cost); the numbers here judge response content only, symmetric for both sides.

Behavioral checks

Truncation rate at 1024 max tokens: 0.03% (1/2970)
Turn-end probability probe: P(<|im_end|>) = 0.92
No user-turn leakage, no runaway generations

Limitations

No preference optimization — no DPO/RLHF; relative quality gaps remain against RLHF'd instruct models
Style transfer — speaks Llama-3.1-405B-style prose (teacher stamp), not Qwen-style
No thinking mode, no tool/function calling — those subsets were excluded
LLM-judge evaluation only — GLM-5.3 as judge; human preferences may differ
Trained for general chat at ≤2048 tokens; long-context behavior untested

License

Apache-2.0 (inherited from the base model). The training data is a mixture of SmolTalk2 subsets, each with its own upstream license terms — see the smoltalk2 dataset card and the original sources linked there.

Credits

Qwen team — the base model
HuggingFaceTB — the SmolTalk2 dataset and the SmolLM3 recipe this builds on
TRL — training framework

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

ziliangpeng

Model Tree

Base

Qwen/Qwen3-4B-Base

Fine-tuned

this model

Input Modalities

Text

Output Modalities