GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
W-Kaski
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
W-Kaski
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
This adapter generates one SQL query from an explicit SQL schema and one natural-language question. It was developed as part of a reproducible parameter-efficient fine-tuning benchmark with LoRA rank ablations, normalized Exact Match evaluation, SQL parsing checks, structured error analysis, controlled SQLite execution tests, and a local API demo.
This model is intended for controlled experiments and demonstrations. It is not a production database agent.
bash
pip install torch transformers peft accelerate safetensorsA GPU is recommended.
Some hosted notebook environments may include an incompatible preinstalled version of torchao. If adapter loading fails with a torchao version error, remove the incompatible package and restart the runtime:
bash
pip uninstall -y torchaopython
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer BASE_MODEL_ID = "Qwen/Qwen2.5-1.5B-Instruct"ADAPTER_ID = "W-Kaski/qwen25-15b-text2sql-lora-r32" tokenizer = AutoTokenizer.from_pretrained( BASE_MODEL_ID, use_fast=True,) if tokenizer.pad_token is None: tokenizer.pad_token = tokenizer.eos_token base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL_ID, torch_dtype=torch.float16, device_map="auto",) model = PeftModel.from_pretrained( base_model, ADAPTER_ID,)model.eval() def build_prompt(schema: str, question: str) -> str: return ( "You are a Text-to-SQL assistant.\n" "Use the SQL schema to answer the question with one SQL query.\n\n" f"Schema:\n{schema.strip()}\n\n" f"Question:\n{question.strip()}\n\n" "SQL:" ) schema = """CREATE TABLE employees ( id INTEGER, name TEXT, department TEXT, salary INTEGER);""" question = ( "What is the average salary of employees " "in the engineering department?") prompt = build_prompt(schema, question)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.inference_mode(): output_ids = model.generate( **inputs, max_new_tokens=128, do_sample=False, pad_token_id=tokenizer.eos_token_id, ) generated_ids = output_ids[0, inputs["input_ids"].shape[1]:]sql = tokenizer.decode(generated_ids, skip_special_tokens=True).strip()print(sql)Example output:
sql
SELECT AVG(salary)FROM employeesWHERE department = 'engineering';Generated SQL may differ textually while remaining semantically equivalent.
The adapter was trained with the following prompt format:
text
You are a Text-to-SQL assistant.Use the SQL schema to answer the question with one SQL query.Schema:<SQL schema>Question:<natural-language question>SQL:Using a substantially different prompt format may reduce performance.
Appropriate uses include:
This adapter is not intended for:
Qwen/Qwen2.5-1.5B-Instructb-mc2/sql-create-contextanswer, question, contextThe main evaluation uses a fixed 500-example split.
| Model | Normalized Exact Match | SQL Parse Valid |
|---|---|---|
| Base model | 0.044 | 0.980 |
| LoRA rank 8 | 0.684 | 0.992 |
| LoRA rank 16 | 0.696 | 0.994 |
| LoRA rank 32 | 0.712 | 0.990 |
Normalized Exact Match compares normalized SQL strings. It may reject semantically equivalent SQL queries.
SQL Parse Valid measures whether generated output can be parsed as SQL. It does not establish semantic or execution correctness.
A separate set of 30 manually curated SQLite cases was used as an execution sanity check.
| Metric | Value |
|---|---|
| Cases | 30 |
| Parse-valid rate | 1.000 |
Read-only SELECT rate | 1.000 |
| Execution-valid rate | 1.000 |
| Execution accuracy | 0.600 |
| P50 latency | 0.456 s |
| P95 latency | 1.182 s |
These 30 cases are not a held-out benchmark and should not be interpreted as general Text-to-SQL accuracy.
Observed failure modes include:
NULL handling;GROUP BY or HAVING clauses;LIMIT and OFFSET errors;The primary dataset is dominated by relatively simple, mostly single-table SQL. Performance should not be extrapolated to complex multi-table enterprise schemas.
Do not directly execute generated SQL against a production database.
At minimum, use:
The accompanying GitHub repository includes a controlled SQLite execution sandbox and a guarded FastAPI demo.
Training, evaluation, error analysis, API code, SQLite checks, and configuration files are available in the project repository:
https://github.com/W-Kaski/qwen-qlora-sql-benchmark
The repository includes:
Full rank-ablation results were produced on a local WSL GPU. Kaggle validation covered environment setup, tests, lint, dataset preparation, and GPU runtime availability, not the reported full training runs.
This adapter is released under the Apache 2.0 license. The base model and training dataset retain their respective licenses and terms.
SELECT AVG(salary)FROM employeesWHERE department = 'engineering';