GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
maxdemarzi
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
maxdemarzi
Model Tree
Qwen/Qwen2.5-Coder-3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Generation must use ARCTIC_SYSTEM, shipped here as system_prompt.txt:
You are a data science expert. Below, you are provided with a database schema and a natural language question. Your task is to understand the schema and generate a valid SQL query to answer the question.
with the Arctic user template (engine, db_details, question, output_format),
output_format = the terse fenced-SQL block, not the <think>/<answer>
reasoning scaffold. The rung is duckdb-values: evidence, comments and sample
values on, reasoning off.
A different system prompt is not a small degradation. The sibling PyRel adapter records that dropping its own system message makes the model emit a different language entirely, which reads as a broken model rather than a missing message.
| decoding | greedy, temperature=0 |
max_new_tokens | 4096 at evaluation — median output is 165 characters, but the long tail reached 13,598 |
| extractor | first ```sql fence, then strip the leading sql tag |
Because reasoning is off there is exactly one fence, so first and last coincide. That distinction matters for reasoning models on this same pipeline, where taking the first fence scores a draft the model went on to correct.
BIRD holdout, 428 scoreable questions, paired against the same 3B fine-tuned to emit PyRel directly:
| arm | correct | paired vs PyRel | |
|---|---|---|---|
| 3B → PyRel direct | 165/428 | 38.6% | — |
| 3B → SQL → transpiler (this adapter) | 179/428 | 41.8% | +60/−46, p = 0.21 |
| routed: SQL where it fires, PyRel where it declines | 206/428 | 48.1% | +60/−19, p = 4.2e-06 |
The head-to-head is not significant. This adapter is not measurably better than emitting PyRel directly; the gain is in routing, because the two arms fail on different questions. Deploy it as one arm of a pair, or accept 41.8% knowing 33% of questions get no program at all.
Decomposition: SQL accuracy 44.3% × transpiler carry 94.2% = 41.7% against a measured 41.8%. Zero malformed outputs in 445 generations.
Re-running all 445 predictions through synthesize_ast:
markdown
302 admitted (ast_join 165, ast_single 54, ast_join_n 36, ast_bound 26, +6 patterns)143 no template0 raised
no template is a transpiler gap, not a model error — the SQL may be perfectly
correct. Each one is a deterministic, permanently fixable Python bug, which is the
improvement path this arm has and a direct-to-PyRel model does not.
sqlglot is a hard dependency and its absence is silent. synthesize_ast
catches the ImportError and returns None, which is also how it signals a
decline — so a deployment without sqlglot declines 100% of requests and looks like
a transpiler that cannot handle your queries. Verified: it returned 60 of 60
no template on a machine that was simply missing the package.
Trained on BIRD train only. Evaluated on the four BIRD dev databases held out from
training (california_schools, debit_card_specializing, superhero,
thrombosis_prediction).
The recipe is pinned to an RTX 5090 deliberately: an identical recipe on different hardware has moved an adapter by 11 questions on this project, which is larger than the effect being measured here.