cwolff

qwen3.5-2b-grpo-synsql

README

License: apache-2.0

Results — BIRD dev

Measured on this repo as published, loaded straight into vLLM. n=300 BIRD dev rows (seeded), 6-turn agentic episodes, T=0.6 / top-p 0.95 / top-k 20, scored by executing against the real SQLite databases. execution_accuracy_official is BIRD-leaderboard set semantics; strict is order- and duplicate-sensitive. full puts the DDL in the prompt; withheld gives the model only the question and makes it discover the schema through tools.

Table with columns: stock Qwen3.5-2B, this model, stock Qwen3.8-27B
	stock Qwen3.5-2B	this model	stock Qwen3.8-27B
BIRD `full`, official	0.290	0.460	0.687
BIRD `full`, strict	—	0.417	—
BIRD `withheld`, official	0.193	0.443	0.693
BIRD `withheld`, strict	—	0.400	—

BIRD dev accuracy across the five models

All five bars were produced by the same harness in the same way, which is what makes them comparable. Two things the chart says out loud:

RL on a 2B works, and works hardest where the schema is hidden. Against its own starting point the gain is +0.170 with the schema in the prompt and +0.250 with it withheld -- exploration is where the training pays.
A 13x larger untrained model still wins. Stock Qwen3.8-27B reaches 0.687 / 0.693 without any training on this task. If you have the VRAM for a 27B and do not need a 2B's latency or cost, it is the better model; these checkpoints are for the case where you need the small one.

Episode behaviour: submit rate 0.31 / 0.20 (full/withheld), generated SQL executes 93.7% / 96.0% of the time, 5.3 / 5.8 mean turns.

On the stock-2B column. It is a fresh run of this harness against Qwen/Qwen3.5-2B, not the figure the trainer logged at step 0 (which was 0.13-0.16 on full). The two disagree, and the cause is not settled: the behavioural counters match closely but the fallback-to-last-working-query fires on 7% of episodes here against 41% there, and an untrained model's score is dominated by that fallback. The trained models barely use it (2-7%), which is why their numbers reproduce their training records to within a point or two. The honest reading is that the 0.290 figure is the one measured the same way as every other bar in the chart, and that an untrained baseline in this harness carries more measurement uncertainty than the trained ones do.

Does the published repo match what training measured?

Training could not measure this repo. Its vLLM engine was built from the base model with the checkpoint's weights synced in — a workaround forced by the fact that the raw training export is not loadable by vLLM at all. This repo is the rebuilt, directly-loadable one, so "same model" needed checking rather than assuming.

Both paths were run over the same 300 questions and compared paired (exact McNemar + bootstrap CI), which is the right test here because at n=300 the unpaired noise floor is several points:

Table with columns: mode, weight-sync path, this repo, paired delta, McNemar p
mode	weight-sync path	this repo	paired delta	McNemar p
`full`	0.457	0.460	+0.003, 95% CI [-0.043, +0.053]	1.00
`withheld`	0.393	0.443	+0.050, 95% CI [+0.007, +0.093]	0.036

full shows no difference. withheld does: the published path scored 5 points HIGHER, and that one clears p<0.05 on its own. Two reasons not to read it as a real gap between the paths. It is one significant result out of six such tests run across these three models, which is roughly what chance alone produces at alpha=0.05 (it does not survive correcting for the six). And this model truncates 35-51% of its episodes, so its run-to-run variance is far larger than its siblings'. The weights in this repo are bit-identical to the training checkpoint's -- that was verified tensor by tensor, not inferred -- so there is no mechanism for the published model to be better. Treat both paths as the same model measured twice.

Two caveats that apply to the absolute numbers regardless:

The checkpoint was selected on BIRD. Steps 1250-1650 all sit in a 0.41-0.47 band, so treat ~0.45 full as the plateau rather than any single step as a clean result.
It is the weakest of the three, and that gap is real. Its siblings reach ~0.55 full against this model's 0.46 -- roughly 9 points, which is well outside the measurement floor, unlike the ~1 point separating the other two from each other. See the training-data limitation below for the most likely reason.

Numbers are BIRD dev, produced by this project's own harness, and are not leaderboard submissions.

Serving with vLLM

The repo is a complete Qwen3_5ForConditionalGeneration checkpoint, so it loads directly — no conversion, no --hf-overrides:

bash
vllm serve cwolff/qwen3.5-2b-grpo-synsql --max-model-len 16384

python
from vllm import LLM, SamplingParams
llm = LLM(model="cwolff/qwen3.5-2b-grpo-synsql", max_model_len=16384)
out = llm.chat(
    [{"role": "system", "content": SYSTEM_PROMPT},
     {"role": "user", "content": "How many singers are there?"}],
    SamplingParams(temperature=0.6, top_p=0.95, top_k=20, max_tokens=2048),
)

How to prompt it

This model expects an agent loop, and will underperform badly without one. It was trained under the exec_plan protocol: each assistant turn emits a single JSON object and nothing else — no prose, no code fences — and your harness executes it and feeds the result back as the next user turn.

json
{"tool": "list_tables"}
{"tool": "describe_table", "args": {"table_name": "singer"}}
{"tool": "run_query", "args": {"sql": "SELECT COUNT(*) FROM singer"}}
{"tool": "submit_sql", "args": {"sql": "SELECT COUNT(*) FROM singer"}}

Tools: list_tables, describe_table, foreign_keys, join_path, sample_rows, run_query, distinct_values, submit_sql. submit_sql ends the episode. Assistant turns open with a <think> block — keep thinking enabled; the chat template does this by default.

The exact system prompt matters. It is EXEC_PLAN_TOOL_INSTRUCTIONS in shared/tools/protocol.py of the SQaLe finetune repo — use that string rather than paraphrasing the tool list above, which is a summary.

Training

Table

Base	`Qwen/Qwen3.5-2B`, stock (no SFT warm start)
Algorithm	GRPO, token-level loss, sum/1024 aggregation, no advantage normalisation
KL	k3 estimator toward a frozen reference, target 0.005
Clipping	sequence-level importance ratio from vLLM sampler logprobs, 0.2 / 0.28 (DAPO)
Batch	18 questions × 8 candidates = 144 episodes/step, 1 inner epoch
LR	1e-5, cosine, 5% warmup
Reward	dense; 3.0 semantic equivalence, F1 partial credit (scale 0.9), 0.1 join bonus, 0.1 consensus bonus
Environment

Reward is computed by executing the candidate against a real SQLite database built from the row payloads and comparing to the gold result — not by string match against a reference query.

Limitations

2B parameters. It is a small model and will lose to a frontier model on hard multi-join questions.
Trained and evaluated on SQLite only.
Trained at a 6-turn budget with a 12288-token episode cap. Longer budgets are untested and truncation reached 35-51% at this one.
Single-turn "schema in, SQL out" prompting is out of distribution — the withheld numbers above are what this model is actually for.
Inherits the base model's vision tower, which is carried over untouched and was never trained or evaluated here. Treat it as a text model.
Its training databases were nearly empty. A preflight over the SynSQL subset used here measured a median of 2 rows per table, against 68 for SQaLe3 and 927 for BIRD train. Reward is execution-based, so with two rows per table many different queries return the same trivial result and the signal that separates a correct query from a lucky one is largely absent. This is the most likely cause of the gap to its siblings.
It burns tool calls. Mean tool calls per episode rose from ~5 to ~50 over training (peaking near 65), with truncation reaching 32-47%, while accuracy stayed flat. Budget for long episodes, or cap turns.

Provenance

Slurm job 26662810, output grpo1800_synsql, checkpoint best_bird/ (step 1600), selected by the trainer on BIRD full official accuracy. Exported by merging the run's 320 trained language-model tensors with the base model's vision tower and MTP head, under the base config.json.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

cwolff

Model Tree

Base

Qwen/Qwen3.5-2B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality