GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
zaninip
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
zaninip
Model Tree
Qwen/Qwen3-1.7B
Input Modalities
Output Modalities
Supported Functionality
Test split of 490 questions from 7 question templates that never appear in training (the split is made by template, not by example). Execution accuracy: the query is right when its result equals the result of the reference query (rows compared as multisets, numbers to a relative 1e-4).
| Model | Execution accuracy | Valid SQL |
|---|---|---|
| Base, zero-shot | 14.3 % | 86.9 % |
| Base, few-shot (9 train examples; mean of 3 draws, range 34.1-43.1 %) | 37.1 % | 79.0-90.0 % |
| Fine-tuned (this adapter, float16) | 64.1 % | 76.9 % |
| Fine-tuned, GGUF Q4_K_M (CPU, llama.cpp) | 62.0 % | 75.9 % |
| Base, GGUF Q4_K_M | 11.6 % | 73.3 % |
The fine-tuned model beats zero-shot on 246 questions against 2, and every few-shot draw (p < 1e-12, paired McNemar). By language: French 70.2 %, Italian 58.0 %. Quantization to Q4_K_M costs 2 points, not significant (p = 0.36).
What fine-tuning taught is the domain's conventions, written in the prompt but ignored by the base model: power in MW turned into energy (MWh = sum of MW x 0.5 h), "renewable" and "low-carbon" as defined sources, peak hours and seasons, regional aliases ("PACA", "Alta Francia" for Hauts-de-France), running sums, national totals over the 12 regions. Of 21 sampled zero-shot errors, 16 are fixed; invented filters and unit slips disappear.
The model expects the exact system prompt it was trained with: instructions, the table schema and the glossary of conventions, 4,528 tokens. Without it the model does not know the table. The prompt is built by the project's code:
bash
git clone https://github.com/zaninip/domain-text2sql && cd domain-text2sqluv sync --extra modelpython
from pathlib import Path import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer from t2sql.eval.extract import extract_sqlfrom t2sql.eval.run import format_promptfrom t2sql.prompts import system_prompt tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B")model = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen3-1.7B", dtype=torch.float16, device_map="auto")model = PeftModel.from_pretrained(model, "zaninip/qwen3-1.7b-eco2mix-sql").merge_and_unload() system = system_prompt(Path("domains/eco2mix"))prompt = format_prompt(tokenizer, system, "Production éolienne en Bretagne en 2023, en GWh ?")inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate(**inputs, do_sample=False, max_new_tokens=256)answer_ids = output[0, inputs["input_ids"].shape[1] :] # the generated part onlyprint(extract_sql(tokenizer.decode(answer_ids, skip_special_tokens=True)))format_prompt applies Qwen3's chat template with enable_thinking=False, as in training.
Decoding is greedy. Run the SQL on the éCO2mix DuckDB file built by make data; the project's
t2sql.db executes model output read-only, one SELECT only, with a timeout.
unsloth/Qwen3-1.7B-unsloth-bnb-4bit); the adapter is evaluated and
published on the original float16 weights. LoRA r = 16, alpha = 32, dropout 0, on all linear
layers (17.4 M trainable parameters). Learning rate 2e-4, cosine schedule, 5 % warmup, batch
1 x 16 gradient accumulation, AdamW 8-bit, fp16. Loss on the SQL answer only.This adapter can be the starting point of more training, for instance on questions it gets wrong. Three conditions matter:
t2sql.train.sft.prepare_example does exactly this for a chat record
{"messages": [system, user, assistant]}; t2sql.prompts.chat_messages builds such a
record.make data and
make dataset, and add the new examples to it.python
import torchfrom peft import PeftModel, prepare_model_for_kbit_trainingfrom transformers import AutoModelForCausalLM, BitsAndBytesConfig quantization = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16)base = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen3-1.7B", quantization_config=quantization, device_map="auto")base = prepare_model_for_kbit_training(base)model = PeftModel.from_pretrained(base, "zaninip/qwen3-1.7b-eco2mix-sql", is_trainable=True)# then: examples from prepare_example, and a Trainer (or TRL's SFTTrainer, packing off)With a larger dataset, a fresh run from the base model with the project's
python -m t2sql.train.sft (about 4 hours on a T4) is often cleaner and easier to evaluate
honestly. Either way, keep a test set of templates the model never trained on.
WHERE, LIMIT without the GROUP BY it needs), hence a
valid-SQL rate below the base model's.A demonstration of what fine-tuning a small model brings to a narrow text-to-SQL task, and a starting point for similar work. Model-generated SQL is untrusted: run it read-only, with a single-statement check and a timeout, as the project does.
git clone https://github.com/zaninip/domain-text2sql && cd domain-text2sqluv sync --extra modelfrom pathlib import Path import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer from t2sql.eval.extract import extract_sqlfrom t2sql.eval.run import format_promptfrom t2sql.prompts import system_prompt tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B")model = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen3-1.7B", dtype=torch.float16, device_map="auto")model = PeftModel.from_pretrained(model, "zaninip/qwen3-1.7b-eco2mix-sql").merge_and_unload() system = system_prompt(Path("domains/eco2mix"))prompt = format_prompt(tokenizer, system, "Production éolienne en Bretagne en 2023, en GWh ?")inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate(**inputs, do_sample=False, max_new_tokens=256)answer_ids = output[0, inputs["input_ids"].shape[1] :] # the generated part onlyprint(extract_sql(tokenizer.decode(answer_ids, skip_special_tokens=True)))import torchfrom peft import PeftModel, prepare_model_for_kbit_trainingfrom transformers import AutoModelForCausalLM, BitsAndBytesConfig quantization = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16)base = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen3-1.7B", quantization_config=quantization, device_map="auto")base = prepare_model_for_kbit_training(base)model = PeftModel.from_pretrained(base, "zaninip/qwen3-1.7b-eco2mix-sql", is_trainable=True)# then: examples from prepare_example, and a Trainer (or TRL's SFTTrainer, packing off)