GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Amr-Kh-2004
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Amr-Kh-2004
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Evaluated with a custom execution-accuracy metric (does the generated SQL run and return the correct data, checked against an in-memory SQLite database - not a string comparison) over 500 held-out test examples from gretelai/synthetic_text_to_sql:
| Model | Execution accuracy |
|---|---|
| Base (no adapter) | 7.27% |
| This adapter | 67.67% |
Breakdown by query complexity - the base model scores 0% on any query needing GROUP BY, JOIN + GROUP BY, or ORDER BY/LIMIT; fine-tuning brings these to 58–76%:
Full methodology, the evaluation code, and training notebook: see the GitHub repository
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModelbase_model_name = "Qwen/Qwen2.5-1.5B-Instruct"adapter_repo = "Amr-Kh-2004/Qwen2.5-1.5B-Instruct-text2sql-lora"quantization_config = BitsAndBytesConfig(load_in_4bit=True,bnb_4bit_quant_type="nf4",bnb_4bit_compute_dtype=torch.float16,)tokenizer = AutoTokenizer.from_pretrained(adapter_repo)base_model = AutoModelForCausalLM.from_pretrained(base_model_name,device_map="auto",quantization_config=quantization_config,torch_dtype=torch.float16,)model = PeftModel.from_pretrained(base_model, adapter_repo)model.eval()question = "List all employees hired after 2020."db_schema = "CREATE TABLE employees (id INT, name VARCHAR(50), hire_date DATE);"chat = [{"role": "system", "content": "You are an SQL query generator given a question and a database schema, generate an SQL query."},{"role": "user", "content": f"Question: {question}\nDatabase Schema: {db_schema}\nSQL Query:"},]prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device)input_len = inputs["input_ids"].shape[1]output = model.generate(**inputs, max_new_tokens=256, do_sample=False)print(tokenizer.decode(output[0][input_len:], skip_special_tokens=True))
FastLanguageModel
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_projgretelai/synthetic_text_to_sql's train splitJOIN+GROUP BY queries in particular would likely improve with more training data/steps.collections.Counter), so it does not check row order - ORDER BY/LIMIT accuracy may be modestly overstated.