GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Bhuvandesai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Bhuvandesai
Model Tree
microsoft/Phi-3-mini-4k-instruct
Input Modalities
Output Modalities
Supported Functionality
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base = "microsoft/Phi-3-mini-4k-instruct"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "Bhuvandesai/phi3-text-to-sql-adapter") SCHEMA = """You are a Text-to-SQL generator. Given a database schema and a natural languagequestion, write a valid SQLite query. Output only the raw SQL. Database Schema:Table departments(id, name, manager_id)Table employees(id, name, department_id, salary, hire_date, manager_id)Table products(id, name, category, price)Table sales(id, employee_id, product_id, amount, quantity, sale_date)""" msgs = [{"role": "user", "content": f"{SCHEMA}\n\nQuestion: What is the average salary by department?"}]prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=128, do_sample=False)print(tok.decode(out[0], skip_special_tokens=True))For CPU / no-GPU use, prefer the quantized GGUFs with llama.cpp (see the GGUF repo).
| Method | QLoRA (4-bit NF4 + double-quant, bf16 compute) |
| LoRA | r=8, α=16, dropout=0.05, bias=none |
| Trainable params | 4,456,448 (0.1165% of 3.82B) |
| Data | 50 train / 12 held-out NL→SQL pairs (synthetic schema) |
| Schedule | 3 epochs, effective batch 4, lr 2e-4 cosine, paged_adamw_8bit |
| Hardware | NVIDIA RTX 4050 Laptop (6 GB) |
| Runtime / peak VRAM | 193.7 s / 5.21 GB reserved |
| Metric | Base Phi-3-mini | This adapter |
|---|---|---|
| Execution-match (run SQL, compare rows) | 41.7% | 75.0% |
| Valid SQL rate | 100% | 100% |
| Eval loss (end of training) | — | 0.0597 (−89.9%) |
| Eval token accuracy | — | 98.4% |
Strict execution-match is conservative: 2 of the 3 held-out "misses" are reasonable answers with a different column projection than the reference; counting "query correctly answers the question" ≈ 92%.
q/k/v (qkv_proj) and gate/up (gate_up_proj), PEFT name-matching adapted only o_proj and down_proj (2 of the 7 listed modules). It still trained well; a future version should target qkv_proj/gate_up_proj for fuller coverage.A full write-up (fine-tuning + quantization deep dive with all benchmarks) accompanies this model.
License: MIT (inherits from the base model).
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base = "microsoft/Phi-3-mini-4k-instruct"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "Bhuvandesai/phi3-text-to-sql-adapter") SCHEMA = """You are a Text-to-SQL generator. Given a database schema and a natural languagequestion, write a valid SQLite query. Output only the raw SQL. Database Schema:Table departments(id, name, manager_id)Table employees(id, name, department_id, salary, hire_date, manager_id)Table products(id, name, category, price)Table sales(id, employee_id, product_id, amount, quantity, sale_date)""" msgs = [{"role": "user", "content": f"{SCHEMA}\n\nQuestion: What is the average salary by department?"}]prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=128, do_sample=False)print(tok.decode(out[0], skip_special_tokens=True))