Results: BIRD dev (execution result accuracy)
All models use a single chain-of-thought sample unless noted, so the comparison is like for like.
Table with columns: Model, Params, Result accuracy| Model | Params | Result accuracy |
|---|
| Base Qwen2.5-Coder-7B-Instruct | 7B | 27.0% |
| This model, greedy | 7B | 52.1% |
| This model, self-consistency (K=8) | 7B | 58.5% |
| DeepSeek V4-Pro | 1.6T | 58.7% |
| GLM 5.2 | 744B | 63.0% |
Result accuracy is the fraction of queries whose SQL executes to the same rows as the gold query (BIRD's official execution metric). The larger frontier models score higher, as expected. The point is that a 7B reaches their accuracy band at a fraction of the size and cost. The 58.5% figure uses K=8 self-consistency (roughly 8x inference).
Usage
Prompt the model to reason step by step. It returns the reasoning followed by a fenced SQL block. Take the last SQL block as the query.
import re, torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "jk200201/qwen2.5-coder-7b-bird-cot"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")
SYSTEM = ("You are an expert SQLite query writer. Reason step by step about the schema "
"and the question, then output the final query in a fenced sql code block.")
def build_prompt(schema, question):
return ("Given the database schema and question, work out the correct SQLite query step by step.\n\n"
f"Database Schema:\n{schema}\n\nQuestion: {question}\n\n"
"Think step by step, then give the final answer in a fenced sql block.")
messages = [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": build_prompt(
"CREATE TABLE singer (Singer_ID INT, Name TEXT, Age INT);",
"How many singers are older than 40?")},
]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
resp = tok.decode(out[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True)
sql = re.findall(r"```(?:sql)?\s*(.*?)```", resp, re.DOTALL)[-1].strip()
print(sql)
For best accuracy (58.5%), sample K=8 at temperature 0.8, execute each candidate, and take the majority result (self-consistency).
Model family
Table with columns: Artifact, Repo| Artifact | Repo |
|---|
| Merged model (this repo) | jk200201/qwen2.5-coder-7b-bird-cot |
| LoRA adapter | jk200201/qwen2.5-coder-7b-bird-cot-lora |
| Training data | jk200201/bird-cot-sft |
Training
Reasoning distillation (CoT-SFT). A strong teacher (Qwen3-Coder-480B) generated step-by-step CoT solutions on BIRD train. Only execution-verified-correct chains were kept (5,593 examples), then supervised fine-tuned into the 7B. Distilling the teacher's reasoning generalized across BIRD's cross-domain dev databases better than distilling SQL answers directly.
Config: QLoRA (4-bit NF4, LoRA rank 32, alpha 64), 2 epochs, learning rate 2e-4 cosine, max sequence length 8192.
Limitations
- Tuned for BIRD-style analytic SQL over realistic schemas. Unusual dialects or domains may need adaptation. It emits SQLite dialect.
- Greedy (52.1%) is the deployable single-shot number. The 58.5% figure needs K=8 self-consistency (roughly 8x inference).
- This is a 7B model. Review generated SQL before running it on production data.
Citation
@misc{qwen25coder7b_bird_cot_2026,
title = {Qwen2.5-Coder-7B BIRD CoT: reasoning distillation for text-to-SQL},
author = {Jenish Kothari},
year = {2026}
}
Acknowledgements
Base model: Qwen2.5-Coder by Alibaba Qwen. Teacher: Qwen3-Coder-480B. Benchmark: BIRD (bird-bench.github.io).