GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
krishy-d
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
krishy-d
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Contemporary instruction-tuned LLMs (including Qwen 2.5) over-format their responses by default — producing bulleted lists, bold section headers, and templated structures even when flowing prose would serve the reader better. This adapter nudges the base model toward prose responses on contexts where prose is appropriate, while preserving the base model's ability to use structure where structure genuinely helps.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerimport torch BASE = "Qwen/Qwen2.5-1.5B-Instruct"ADAPTER = "krishy-d/prosify_qwen_1.5b_lora" tokenizer = AutoTokenizer.from_pretrained(BASE)base_model = AutoModelForCausalLM.from_pretrained( BASE, dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base_model, ADAPTER) # Generatemessages = [{"role": "user", "content": "Write me an email to my manager about WFH tomorrow."}]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): output = model.generate(**inputs, max_new_tokens=200, do_sample=False, pad_token_id=tokenizer.pad_token_id)print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))| Base model | Qwen/Qwen2.5-1.5B-Instruct |
| Training method | DPO (Direct Preference Optimization) |
| Adapter | LoRA |
| Training data | FormatBench train split (~440 examples) |
| Validation | FormatBench val split (~50 examples) |
| LoRA rank | 16 |
| LoRA alpha | 32 |
| Target modules | q_proj, k_proj, v_proj, o_proj |
| Learning rate | 5e-5 |
| Epochs | 1 |
| β (KL leash) | 0.1 |
| Effective batch size | 4 |
| Precision | bfloat16 |
| Hardware | Kaggle T4 GPU (free tier) |
Full training notebook: notebooks/dpo_02_train.ipynb.
Structural metrics computed on the FormatBench held-out test split (49 examples, gold = prose) and the adversarial held-out set (40 examples, gold = structure).
| Metric | Base model | This model | Gold response |
|---|---|---|---|
| Bullets per response | 2.16 | 0.53 | 0.00 |
| Headers per response | 0.59 | 0.00 | 0.00 |
The trained adapter reduces bullet usage by 75% and eliminates markdown headers on prose-appropriate contexts.
| Metric | Base model | This model | Gold response |
|---|---|---|---|
| Bullets per response | 9.35 | 6.60 | 8.18 |
| Headers per response | 0.78 | 0.30 | 3.38 |
On contexts where structure is the correct answer (recipes, install instructions, comparisons, troubleshooting flows, reference lookups), the trained model preserves substantial structure but drifts below the gold level — indicating mild reward hacking where the model slightly over-generalizes the "prefer prose" preference.
Full evaluation notebook: notebooks/dpo_03_evaluate.ipynb.
chosen responses were authored
by one annotator, so the trained model inherits that voice as the "correct" prose
style.This adapter is v1. Planned for v2:
bibtex
@misc{prosify_qwen_1_5b_lora_2026, author = {Krishna Dahale}, title = {Prosify-Qwen-1.5B-LoRA: A DPO-trained adapter for correcting LLM formatting bias}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/krishy-d/prosify_qwen_1.5b_lora}}}Apache 2.0 (matching the base model's license).
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerimport torch BASE = "Qwen/Qwen2.5-1.5B-Instruct"ADAPTER = "krishy-d/prosify_qwen_1.5b_lora" tokenizer = AutoTokenizer.from_pretrained(BASE)base_model = AutoModelForCausalLM.from_pretrained( BASE, dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base_model, ADAPTER) # Generatemessages = [{"role": "user", "content": "Write me an email to my manager about WFH tomorrow."}]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): output = model.generate(**inputs, max_new_tokens=200, do_sample=False, pad_token_id=tokenizer.pad_token_id)print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))@misc{prosify_qwen_1_5b_lora_2026, author = {Krishna Dahale}, title = {Prosify-Qwen-1.5B-LoRA: A DPO-trained adapter for correcting LLM formatting bias}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/krishy-d/prosify_qwen_1.5b_lora}}}