GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
pashoktond1
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
pashoktond1
Model Tree
Qwen/Qwen2.5-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel base = "Qwen/Qwen2.5-7B-Instruct"adapter = "pashoktond1/qwen2.5-7b-deal-lora" bnb = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,)model = AutoModelForCausalLM.from_pretrained(base, quantization_config=bnb, device_map="auto")model = PeftModel.from_pretrained(model, adapter)tokenizer = AutoTokenizer.from_pretrained(base) SYSTEM = ("You write DEAL mini-apps. Produce a .deal file (state types, action classes, " "@ui-update handlers) and a .dealui file (the view). Output the .deal first, then " "the .dealui, each wrapped in a fenced code block labeled deal and dealui. " "DEAL has no ternary operator (use if/else), no method calls (use functions like " "contains(x, y), length(x)), and `from`/`async` are reserved keywords. Use === and !==.") def generate(prompt, temperature=0.7): msgs = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": prompt}] text = tokenizer.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True) inputs = tokenizer(text, return_tensors="pt").to(model.device) out = model.generate(**inputs, max_new_tokens=1536, temperature=temperature, do_sample=True, top_p=0.9) return tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True) print(generate("Build a simple todo list with add and complete"))from as a name, ternary ?:, method calls). Constrained decoding (GBNF) is the intended
fix for those.import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel base = "Qwen/Qwen2.5-7B-Instruct"adapter = "pashoktond1/qwen2.5-7b-deal-lora" bnb = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,)model = AutoModelForCausalLM.from_pretrained(base, quantization_config=bnb, device_map="auto")model = PeftModel.from_pretrained(model, adapter)tokenizer = AutoTokenizer.from_pretrained(base) SYSTEM = ("You write DEAL mini-apps. Produce a .deal file (state types, action classes, " "@ui-update handlers) and a .dealui file (the view). Output the .deal first, then " "the .dealui, each wrapped in a fenced code block labeled deal and dealui. " "DEAL has no ternary operator (use if/else), no method calls (use functions like " "contains(x, y), length(x)), and `from`/`async` are reserved keywords. Use === and !==.") def generate(prompt, temperature=0.7): msgs = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": prompt}] text = tokenizer.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True) inputs = tokenizer(text, return_tensors="pt").to(model.device) out = model.generate(**inputs, max_new_tokens=1536, temperature=temperature, do_sample=True, top_p=0.9) return tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True) print(generate("Build a simple todo list with add and complete"))