GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Rut-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Rut-ai
Model Tree
Qwen/Qwen2.5-0.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen2.5-0.5B-Instruct |
| Fine-tuning Method | QLoRA (4-bit NF4 + LoRA rank 16) |
| Training Data | NCERT Themes in Indian History Part II (128 pages) |
| Training Samples | ~165 text continuation samples |
| Epochs | 5 |
| GPU | NVIDIA RTX 3050 4GB |
| Training Time | ~3.4 minutes |
| Final Train Loss | 2.674 |
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel # Load adapter on top of base modeltokenizer = AutoTokenizer.from_pretrained("Rut-ai/indian-history-qlora")model = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen2.5-0.5B-Instruct", dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "Rut-ai/indian-history-qlora")model.eval() # Generatedef ask(instruction, context=""): if context: prompt = f"### Instruction:\n{instruction}\n\n### Input:\n{context}\n\n### Response:\n" else: prompt = f"### Instruction:\n{instruction}\n\n### Response:\n" inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=200, temperature=0.7, do_sample=True, repetition_penalty=1.1) return tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True) # Exampleprint(ask( "Continue the following passage about Indian history:", "The Mughal Empire was one of the largest empires in Indian history."))The model was trained on text continuation in Alpaca format:
text
### Instruction:Continue the following passage about Indian history: ### Input:[First 150 words of a historical text chunk] ### Response:[Continuation of the passage]Apache 2.0 — based on Qwen2.5-0.5B-Instruct
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel # Load adapter on top of base modeltokenizer = AutoTokenizer.from_pretrained("Rut-ai/indian-history-qlora")model = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen2.5-0.5B-Instruct", dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "Rut-ai/indian-history-qlora")model.eval() # Generatedef ask(instruction, context=""): if context: prompt = f"### Instruction:\n{instruction}\n\n### Input:\n{context}\n\n### Response:\n" else: prompt = f"### Instruction:\n{instruction}\n\n### Response:\n" inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=200, temperature=0.7, do_sample=True, repetition_penalty=1.1) return tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True) # Exampleprint(ask( "Continue the following passage about Indian history:", "The Mughal Empire was one of the largest empires in Indian history."))### Instruction:Continue the following passage about Indian history:### Input:[First 150 words of a historical text chunk]### Response:[Continuation of the passage]