GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
samarthruckstar
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
samarthruckstar
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
PEFT LoRA Adapter (adapter_model.safetensors) (~73.9 MB)
Qwen/Qwen2.5-1.5B-Instructr=16, alpha=16q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_projOllama & llama.cpp GGUF (qwen_whatsapp_q4_k_m.gguf) (~986 MB)
Q4_K_M) for fast CPU/GPU inference.python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel BASE_MODEL_ID = "Qwen/Qwen2.5-1.5B-Instruct"LORA_REPO_ID = "samarthruckstar/qwen-whatsapp-agent" tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_ID)base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL_ID, torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base_model, LORA_REPO_ID)model.eval() prompt = "What is the price, power, and capacity of H-ASM15T Automatic Slipper Making Machine?"messages = [ {"role": "system", "content": "You are the official WhatsApp AI assistant for Mahadyuti Engineering Solution."}, {"role": "user", "content": prompt}] inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt").to(model.device)outputs = model.generate(inputs, max_new_tokens=256, do_sample=False)print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))Download qwen_whatsapp_q4_k_m.gguf:
bash
hf download samarthruckstar/qwen-whatsapp-agent qwen_whatsapp_q4_k_m.gguf --local-dir .Run in Ollama:
bash
ollama create qwen-whatsapp -f Modelfileollama run qwen-whatsapp "Hi, tell me about your slipper making machines"import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel BASE_MODEL_ID = "Qwen/Qwen2.5-1.5B-Instruct"LORA_REPO_ID = "samarthruckstar/qwen-whatsapp-agent" tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_ID)base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL_ID, torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base_model, LORA_REPO_ID)model.eval() prompt = "What is the price, power, and capacity of H-ASM15T Automatic Slipper Making Machine?"messages = [ {"role": "system", "content": "You are the official WhatsApp AI assistant for Mahadyuti Engineering Solution."}, {"role": "user", "content": prompt}] inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt").to(model.device)outputs = model.generate(inputs, max_new_tokens=256, do_sample=False)print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))hf download samarthruckstar/qwen-whatsapp-agent qwen_whatsapp_q4_k_m.gguf --local-dir .ollama create qwen-whatsapp -f Modelfileollama run qwen-whatsapp "Hi, tell me about your slipper making machines"