GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
lalitkarthik
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
lalitkarthik
Model Tree
Qwen/Qwen2.5-3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Judged pairwise by an independent model (Gemini 3.1 Pro) on 60 held-out prompts, with A/B order randomised per prompt to neutralise position bias.
| Comparison | Win rate (ties excluded) |
|---|---|
| v2 vs base Qwen2.5-3B-Instruct | 85.7% (48W/8L/4T) |
| v2 vs v1 (the earlier fine-tune) | 72.4% (42W/16L/2T) |
| v1 vs base | 56.4% (31W/24L/5T) |
The third row is the interesting one: the previous fine-tune beat its base model by only ~56%, which is close to chance. The rebuild is what moved it.
Mechanical counts on the same 60 prompts (no judge involved):
| base | v1 | v2 | |
|---|---|---|---|
Replies containing thou/thee/hast/… | 4 | 2 | 0 |
| Capability-retention checks passed | 6/7 | 5/7 | 5/7 |
Capability retention is a tie between v1 and v2 — they fail different checks. Fine-tuning costs roughly 14 points of general ability either way; that is the honest tax.
| Base | Qwen/Qwen2.5-3B-Instruct |
| Method | QLoRA, 4-bit NF4, double quant |
| LoRA | r=16, alpha=32, dropout 0.05 |
| Target modules | all seven: q,k,v,o,gate,up,down_proj |
| Effective batch | 16 (4 x grad-accum 4) |
| LR / schedule | 2e-4, cosine, 3% warmup |
| Hardware | single Kaggle T4 (free tier) |
| Best checkpoint |
Eval loss bottomed at step 550 and rose afterwards while train loss kept falling, so the shipped weights are that checkpoint rather than the final one.
~4,800 examples generated by two teachers (Groq llama-3.3-70b-versatile and Gemini
2.5 Flash) from a cleaned Project Gutenberg corpus, across four task types: persona
dialogue, modern-to-Victorian rewrite, context-grounded QA, and letters. Filtered to
2,827 by rule checks, near-duplicate removal, and an independent LLM judge keeping
score >= 4.
Two teachers rather than one because distilling from a single model stamps its stylistic tics across the whole dataset.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = "Qwen/Qwen2.5-3B-Instruct"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")model = PeftModel.from_pretrained(model, "lalitkarthik/scholar-3b-v2")msgs = [{"role": "system", "content": "You are a thoughtful scholar of the late nineteenth century."},{"role": "user", "content": "I am uneasy about a decision I must make tomorrow."},]enc = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt",return_dict=True).to(model.device)out = model.generate(**enc, max_new_tokens=200, do_sample=True, temperature=0.8, top_p=0.9)print(tok.decode(out[0][enc["input_ids"].shape[-1]:], skip_special_tokens=True))
| step 550, eval_loss 0.4606 |