hari-krishna-ai

qwen3-8b-text2sql-qlora-v2

README

License: apache-2.0

Two things that will silently break your results

1. Pass enable_thinking=False. Qwen3's chat template renders a finished assistant turn with an empty <think></think> block, so every training target was preceded by one. Without this flag at inference the model is asked to resume from a position it never saw in training, and returns reasoning prose, stray </think> tags and repetition loops. This cost one full GPU evaluation run.

2. Use prompt v2, not a prompt of your own. This adapter was trained against a specific system prompt containing a business glossary and a DATA_AS_OF reference date (fingerprint 4e72cc5f722ce436). Swapping the prompt is worth roughly ±10 pp on its own and makes any comparison meaningless.

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel ADAPTER = "hari-krishna-ai/qwen3-8b-text2sql-qlora-v2-v2" bnb = BitsAndBytesConfig(    load_in_4bit=True,    bnb_4bit_quant_type="nf4",    bnb_4bit_use_double_quant=True,    bnb_4bit_compute_dtype=torch.float16,) tok = AutoTokenizer.from_pretrained(ADAPTER)model = AutoModelForCausalLM.from_pretrained(    "Qwen/Qwen3-8B", revision="b968826d9c46",    quantization_config=bnb, device_map={"": 0},    dtype=torch.float16, attn_implementation="sdpa",)model = PeftModel.from_pretrained(model, ADAPTER).eval() messages = [    {"role": "system", "content": SYSTEM_PROMPT_V2},   # see src/model/prompt_v2.py    {"role": "user", "content": f"Database schema:\n\n{SCHEMA}\n\n"                                f"Question: {question}\n\nPostgreSQL query: /no_think"},]text = tok.apply_chat_template(    messages, tokenize=False, add_generation_prompt=True,    enable_thinking=False,          # <- required)out = model.generate(**tok(text, return_tensors="pt").to(model.device),                     max_new_tokens=512, do_sample=False)

Results

453 held-out questions, benchmark v2, greedy decoding, 4-bit on one Kaggle T4.

Table with columns: metric, base, fine-tuned, + repair
metric	base	fine-tuned	+ repair
strict execution accuracy	43.71 %	68.43 %	70.86 %
projection-tolerant	47.68 %	68.43 %	70.86 %
executable SQL	95.58 %	94.48 %	98.23 %
schema hallucination	0.22 %	3.75 %

By difficulty:

Table with columns: tier, n, base, fine-tuned, Δ
tier	n	base	fine-tuned	Δ
easy	126	92.9 %	100.0 %	+7.1
medium	63	69.8 %	96.8 %	+27.0
hard	144	7.6 %	54.9 %	+47.2

Fine-tuning costs something, and self-correction pays it back. The adapter raises schema hallucination 0.22 % → 3.75 %: fitting the training set makes the model more willing to emit a confident wrong identifier. One repair attempt on detectable failures returns it to 0.66 % and executable SQL to 98.23 %, past the base model's level, while keeping the accuracy gain.

Repair rates are reported as two numbers on purpose — 17 of 25 failures now execute (68.0 % success) but only 11 return the gold rows (44.0 % correctness). A single rate would hide that six repairs turned a crash into a confident wrong answer.

Training

Table

base	`Qwen/Qwen3-8B` @ `b968826d9c46`, 4-bit NF4 + double quant
adapter	LoRA r=16, alpha=32, dropout 0.05, all 7 attention + MLP projections
trainable	43,646,976 of 4,761,498,624 — 0.917 %
loss	completion-only, prompt masked to `-100`
supervised token share	2.41 %
sequence length	2048 (longest example 1,944 tokens, 0 truncated)
optimiser

The loss curve is the clearest evidence iteration 2 worked. Iteration 1 ended at train 0.006 / eval 0.335 — a model that had memorised its 2,133 templates. This one ends at train 0.083 / eval 0.145: it fits the training set less hard and generalises more than twice as well. Loss is still not the result; execution accuracy is.

Why completion-only loss matters here. Each example is ~1,814 prompt tokens and ~45 completion tokens, and the prompt is almost entirely schema — byte-identical across all 2,175 examples. Training on the full sequence would send ~97 % of the gradient into memorising a schema the model is handed at inference. The loss curve would look excellent while the ability you care about barely moved.

Fitting a 2,048-token sequence on a 16 GB T4 needed the lm_head and cross-entropy to run only on the ~45 supervised positions rather than all ~1,860 (logits_to_keep). Their 151k-vocabulary logits plus fp32 copies were ~3 GB per step. The loss is identical; the first attempt without this died with CUDA OOM at step 34 of 136.

Resolved package versions (recorded, not pinned): torch 2.10.0+cu128, transformers 5.17.0, peft 0.21.0, bitsandbytes 0.50.2, accelerate 1.15.0, trl 1.13.0.

Limitations

Schema-specific. It learned this database's conventions, which is most of the gain. It will not transfer to another schema without retraining.
very_hard sits at 13.9 %. Window functions and nested aggregation remain largely unsolved; five new training templates moved the tier off zero and no further.
Every test question came from the same generator as training. Real users write abbreviations, typos and genuinely ambiguous requests. This is the biggest caveat on 70.86 %.
One epoch, one seed, one run. No variance estimate.
Silent wrong answers are the real risk. 84 of the 132 remaining failures execute fine and return the wrong rows, and self-correction cannot see them: there is no error to feed back.
Hallucination is higher than the base model's before repair (3.75 % vs 0.22 %). Run the repair loop.

License

Apache 2.0, inheriting the base model's licence. All training data is synthetic.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

hari-krishna-ai

Model Tree

Base

Qwen/Qwen3-8B

Adapter

this model

Input Modalities

Text

Output Modalities