cwolff

qwen3.5-2b-grpo-bird

README

License: apache-2.0

Results — BIRD dev

Measured on this repo as published, loaded straight into vLLM. n=300 BIRD dev rows (seeded), 6-turn agentic episodes, T=0.6 / top-p 0.95 / top-k 20, scored by executing against the real SQLite databases. execution_accuracy_official is BIRD-leaderboard set semantics; strict is order- and duplicate-sensitive. full puts the DDL in the prompt; withheld gives the model only the question and makes it discover the schema through tools.

Table with columns: stock Qwen3.5-2B, this model
	stock Qwen3.5-2B	this model
BIRD `full`, official	0.133	0.553
BIRD `full`, strict	—	0.513
BIRD `withheld`, official	0.077	0.547
BIRD `withheld`, strict	—	0.510

Episode behaviour: submit rate 0.60 / 0.59 (full/withheld), generated SQL executes 97.7% / 97.0% of the time, 4.1 / 5.1 mean turns.

The stock-model column is the same harness run against the base model at step 0 of training, so the comparison is like-for-like: the gain is from RL, not from the scaffolding.

Does the published repo match what training measured?

Training could not measure this repo. Its vLLM engine was built from the base model with the checkpoint's weights synced in — a workaround forced by the fact that the raw training export is not loadable by vLLM at all. This repo is the rebuilt, directly-loadable one, so "same model" needed checking rather than assuming.

Both paths were run over the same 300 questions and compared paired (exact McNemar + bootstrap CI), which is the right test here because at n=300 the unpaired noise floor is several points:

Table with columns: mode, weight-sync path, this repo, paired delta, McNemar p
mode	weight-sync path	this repo	paired delta	McNemar p
`full`	0.517	0.553	+0.037, 95% CI [-0.007, +0.080]	0.13
`withheld`	0.517	0.547	+0.030, 95% CI [-0.010, +0.070]	0.18

No significant difference in either mode, and the published path never scored lower. Read that as "consistent within about ±5 points", not as proof of equality — failure to reject is not equivalence, and the CI width is what the test can actually exclude.

Two caveats that apply to the absolute numbers regardless:

The checkpoint was selected on BIRD. Steps 1250–1650 all sit in a 0.51–0.56 band, so treat ~0.55 full as the plateau rather than any single step as a clean result.
This does not separate from its sibling. The two models differ by ~1 point, well inside the floor. Do not read the training-data comparison off these numbers; run a paired test if you need it.

Numbers are BIRD dev, produced by this project's own harness, and are not leaderboard submissions.

Serving with vLLM

The repo is a complete Qwen3_5ForConditionalGeneration checkpoint, so it loads directly — no conversion, no --hf-overrides:

bash
vllm serve cwolff/qwen3.5-2b-grpo-bird --max-model-len 16384

python
from vllm import LLM, SamplingParams
llm = LLM(model="cwolff/qwen3.5-2b-grpo-bird", max_model_len=16384)
out = llm.chat(
    [{"role": "system", "content": SYSTEM_PROMPT},
     {"role": "user", "content": "How many singers are there?"}],
    SamplingParams(temperature=0.6, top_p=0.95, top_k=20, max_tokens=2048),
)

How to prompt it

This model expects an agent loop, and will underperform badly without one. It was trained under the exec_plan protocol: each assistant turn emits a single JSON object and nothing else — no prose, no code fences — and your harness executes it and feeds the result back as the next user turn.

json
{"tool": "list_tables"}
{"tool": "describe_table", "args": {"table_name": "singer"}}
{"tool": "run_query", "args": {"sql": "SELECT COUNT(*) FROM singer"}}
{"tool": "submit_sql", "args": {"sql": "SELECT COUNT(*) FROM singer"}}

Tools: list_tables, describe_table, foreign_keys, join_path, sample_rows, run_query, distinct_values, submit_sql. submit_sql ends the episode. Assistant turns open with a <think> block — keep thinking enabled; the chat template does this by default.

The exact system prompt matters. It is EXEC_PLAN_TOOL_INSTRUCTIONS in shared/tools/protocol.py of the SQaLe finetune repo — use that string rather than paraphrasing the tool list above, which is a summary.

Training

Table

Base	`Qwen/Qwen3.5-2B`, stock (no SFT warm start)
Algorithm	GRPO, token-level loss, sum/1024 aggregation, no advantage normalisation
KL	k3 estimator toward a frozen reference, target 0.005
Clipping	sequence-level importance ratio from vLLM sampler logprobs, 0.2 / 0.28 (DAPO)
Batch	18 questions × 8 candidates = 144 episodes/step, 1 inner epoch
LR	1e-5, cosine, 5% warmup
Reward	dense; 3.0 semantic equivalence, F1 partial credit (scale 0.9), 0.1 join bonus, 0.1 consensus bonus
Environment

Reward is computed by executing the candidate against a real SQLite database built from the row payloads and comparing to the gold result — not by string match against a reference query.

Limitations

2B parameters. It is a small model and will lose to a frontier model on hard multi-join questions.
Trained and evaluated on SQLite only.
Trained at a 6-turn budget with a 12288-token episode cap. Longer budgets are untested and truncation rose to 8–19% at this one.
Single-turn "schema in, SQL out" prompting is out of distribution — the withheld numbers above are what this model is actually for.
Inherits the base model's vision tower, which is carried over untouched and was never trained or evaluated here. Treat it as a text model.

Provenance

Slurm job 26640987, output grpo1800_bird, checkpoint best_bird/ (step 1350), selected by the trainer on BIRD full official accuracy. Exported by merging the run's 320 trained language-model tensors with the base model's vision tower and MTP head, under the base config.json.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

cwolff

Model Tree

Base

Qwen/Qwen3.5-2B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality