GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
Anurich
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Anurich
Model Tree
Qwen/Qwen2.5-Coder-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Model | BIRD dev EX (149-slice) | Spider dev EX (200) |
|---|---|---|
| Two-stage SFT (start point) | 36.9% | 82.0% |
| + GRPO (this adapter) | 36.2% (54/149) | 83.0% (166/200) |
| Δ | −0.7 | +1.0 |
Both deltas are within the ±~4% noise of these slice sizes. Metric = Execution Accuracy (run predicted SQL against the real SQLite DB, compare result rows to gold). Greedy, single inference, no self-consistency/ensembling.
Because it was earned, and it is informative. We ran GRPO twice, under two different conditions, specifically to rule out the obvious confound:
Attempt #1 — GRPO on Spider-train. The model was already ~82% on Spider,
so most K-sample groups were all-correct → frac_reward_zero_std ≈ 0.7
(70% of groups had zero reward variance → zero advantage → no gradient).
Flat EX. This could be dismissed as "you fed it data it had already
mastered." GRPO's advantage is (r − mean)/std; an all-same-reward group has
std = 0 → no learning signal. This is the textbook GRPO failure mode.
Attempt #2 — GRPO on difficulty-filtered data (this run). We fixed the
confound with a DAPO-style pass-rate filter: sampled the SFT model K=8×
on 1,367 held-out SynSQL questions and kept only the uncertainty band
(0.2 < pass_rate < 0.8). Result: 494 band / 502 all-fail / 184 all-pass
— a healthy, balanced difficulty split. GRPO on the 494 band then trained with
a perfectly healthy mechanism: frac_reward_zero_std fell to 0.05–0.2,
reward_std ≈ 0.44, grad_norm ≈ 0.5, gradient flowing every step.
And the EX still did not move (table above).
Two controlled conditions — flat-reward and healthy-reward — both flat EX. That rules out the reward-variance explanation and points at the method itself: outcome-only GRPO ≈ 0 EX gain on Text-to-SQL. This independently reproduces the Reward-SQL finding (naive outcome-only GRPO 62.6% vs SFT 63.2% — flat-to- negative). The published GRPO gains (~+2.8 EX) require process rewards / a trained PRM, not the outcome-only reward used here.
0.2 < p < 0.8),
execution-verifiable (ship runnable SQLite DBs).GRPOTrainer, 8 generations/prompt, beta=0 (no reference
model), lr=1e-6 (cosine decay), temperature=0.8, epsilon=0.2,
use_vllm=False. 400 steps, ~2h42m on a single 24–32 GB NVIDIA card.1.0 if the SQL executes and the result matches gold; 0.1 if it executes but
is wrong; 0.0 otherwise. Plus a tiny 0.2 format reward for a
block. Sandbox: read-only connection + op-budget interrupt for adversarial SQL.bash
# 1. pass-rate filter (build the uncertainty-zone RL set)python -m slmsql.passrate_filter rs_source.jsonl \--adapter adapters/twostage --k 8 --low 0.2 --high 0.8 --name rl_uncertainty# 2. GRPO on the bandpython -m slmsql.train_grpo_cuda rl_uncertainty.jsonl \--base-adapter adapters/twostage --name grpo2 --num-generations 8# 3. evalpython -m slmsql.evaluate bird_dev_150.jsonl --adapter adapters/grpo2 --name grpo2_bird
```sql