GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
AyushPatel28
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
AyushPatel28
Model Tree
Qwen/Qwen2.5-Coder-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel BASE = "Qwen/Qwen2.5-Coder-7B-Instruct"ADAPTER = "AyushPatel28/PyExplain-qwen-coder-7b" bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16)tok = AutoTokenizer.from_pretrained(BASE)model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")model = PeftModel.from_pretrained(model, ADAPTER) code = "def reverse(s):\n return s[::-1]"msgs = [{"role": "system", "content": "Explain Python code simply and accurately."}, {"role": "user", "content": f"Explain this code:\n```python\n{code}\n```"}]prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)inputs = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)out = model.generate(**inputs, max_new_tokens=300, do_sample=False)print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))