GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
akeakeaki
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
akeakeaki
Model Tree
Qwen/Qwen2.5-Coder-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Qwen/Qwen2.5-Coder-7B-Instructdown_proj, gate_proj, k_proj, o_proj, q_proj, up_proj, v_projpython
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen2.5-Coder-7B-Instruct", dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(base, "akeakeaki/pytorch-rag-lora-synth-v2")tok = AutoTokenizer.from_pretrained("akeakeaki/pytorch-rag-lora-synth-v2") messages = [ {"role": "system", "content": "You are an expert PyTorch assistant. " "Answer using ONLY the provided Context."}, {"role": "user", "content": "Context:\n<retrieved chunks>\n\nQuestion: <q>"},]inputs = tok.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)out = model.generate(inputs, max_new_tokens=400)print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))bash
vllm serve Qwen/Qwen2.5-Coder-7B-Instruct \ --enable-lora \ --lora-modules pytorch-rag=akeakeaki/pytorch-rag-lora-synth-v2 \ --max-lora-rank 16This is a research artifact. In our evaluation the v1 adapter did not
outperform the base model on the RAG task (composite RAG score dropped vs
the base-vanilla baseline) — most likely due to a stylistic shift toward
terse StackOverflow answers and limited training data. See the project
report for the full analysis. Use as a baseline / starting point.
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen2.5-Coder-7B-Instruct", dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(base, "akeakeaki/pytorch-rag-lora-synth-v2")tok = AutoTokenizer.from_pretrained("akeakeaki/pytorch-rag-lora-synth-v2") messages = [ {"role": "system", "content": "You are an expert PyTorch assistant. " "Answer using ONLY the provided Context."}, {"role": "user", "content": "Context:\n<retrieved chunks>\n\nQuestion: <q>"},]inputs = tok.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)out = model.generate(inputs, max_new_tokens=400)print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))