GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Sid9797
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Sid9797
Model Tree
microsoft/Phi-3-mini-4k-instruct
Input Modalities
Output Modalities
Supported Functionality
| Metric | Base Model | Fine-Tuned |
|---|---|---|
| Exact Match | 0.0% | 82.0% |
| Execution Accuracy | 84.0% | 96.0% |
| BLEU Score | 55.79 | 96.42 |
Evaluated on 50 held-out validation examples not seen during training.
| Property | Value |
|---|---|
| Base model | microsoft/Phi-3-mini-4k-instruct (3.8B params) |
| Fine-tuning method | QLoRA (4-bit NF4 + LoRA) |
| LoRA rank | r=16, alpha=32 |
| Trainable parameters | 8,912,896 (0.23%) |
| Training examples | 76,577 |
| Training hardware | NVIDIA RTX 5060 Ti 8GB |
| Training time | 3 hours 2 minutes |
| Final train loss | 0.5677 |
| Max sequence length | 256 tokens |
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel base_model = "microsoft/Phi-3-mini-4k-instruct"adapter = "Sid9797/querycraft-phi3-sql" bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,) tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( base_model, quantization_config=bnb_config, device_map="cuda:0", trust_remote_code=True, torch_dtype=torch.bfloat16,)model = PeftModel.from_pretrained(model, adapter)model.eval() prompt = '''### System:You are a SQL expert. Given a database schema and a natural language question, generate a valid SQL query that answers the question. Output only the SQL query with no explanation. ### Schema:CREATE TABLE employees (id INTEGER, name VARCHAR, department VARCHAR, salary FLOAT) ### Question:What is the average salary by department? ### SQL:''' inputs = tokenizer(prompt, return_tensors="pt").to("cuda")with torch.no_grad(): outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False) sql = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)print(sql.strip().split("\n")[0])# SELECT AVG(salary) FROM employees GROUP BY departmentThe model was trained on the Alpaca instruction format: System: You are a SQL expert. Given a database schema and a natural language question, generate a valid SQL query that answers the question. Output only the SQL query with no explanation. Schema: {CREATE TABLE statements} Question: {natural language question} SQL: {model generates SQL here}
The base Phi-3 Mini, without fine-tuning, consistently wrapped SQL output
in markdown code fences (```sql ... ```) and appended semicolons.
This formatting breaks exact match evaluation even when the SQL logic is correct.
Fine-tuning on consistently formatted examples eliminated this entirely.
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel base_model = "microsoft/Phi-3-mini-4k-instruct"adapter = "Sid9797/querycraft-phi3-sql" bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,) tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( base_model, quantization_config=bnb_config, device_map="cuda:0", trust_remote_code=True, torch_dtype=torch.bfloat16,)model = PeftModel.from_pretrained(model, adapter)model.eval() prompt = '''### System:You are a SQL expert. Given a database schema and a natural language question, generate a valid SQL query that answers the question. Output only the SQL query with no explanation. ### Schema:CREATE TABLE employees (id INTEGER, name VARCHAR, department VARCHAR, salary FLOAT) ### Question:What is the average salary by department? ### SQL:''' inputs = tokenizer(prompt, return_tensors="pt").to("cuda")with torch.no_grad(): outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False) sql = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)print(sql.strip().split("\n")[0])# SELECT AVG(salary) FROM employees GROUP BY department