GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Miladasghari
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Miladasghari
Model Tree
Qwen/Qwen2.5-0.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
safetensors)Because the weights are merged, you do not need the peft library for inference:
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Miladasghari/light-coder" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, dtype=torch.float16, device_map="auto") messages = [ {"role": "user", "content": "Write a Python function to check if a string is a palindrome."}] prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate( **inputs, max_new_tokens=256, temperature=0.3, top_p=0.9, repetition_penalty=1.05) response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)print(response)import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Miladasghari/light-coder" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, dtype=torch.float16, device_map="auto") messages = [ {"role": "user", "content": "Write a Python function to check if a string is a palindrome."}] prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate( **inputs, max_new_tokens=256, temperature=0.3, top_p=0.9, repetition_penalty=1.05) response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)print(response)