GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
adiitya0929
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
adiitya0929
Model Tree
adiitya0929/TinySQL-125M-base
Input Modalities
Output Modalities
Supported Functionality
python
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "adiitya0929/TinySQL-125M-WikiSQL"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo) prompt = ( '<schema>CREATE TABLE t ("Player" TEXT, "No." TEXT, "Nationality" TEXT, "Position" TEXT, ' '"Years in Toronto" TEXT, "School/Club Team" TEXT);</schema>\n' "<question>What position does the player who played for butler cc (ks) play?</question>\n" "<sql>")ids = tok(prompt, return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=64) # greedy; stops at </sql>print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))# SELECT "Position" FROM t WHERE "School/Club Team" = 'butler cc (ks)';Prompt format. The model was trained on exactly this layout. The table is always called t. Column names are double-quoted, and each column is typed REAL (numeric) or TEXT. The question goes in <question>…</question>, and the prompt ends with <sql>. The model writes one SELECT statement and then </sql>.
| Model | Pretraining tokens | Greedy | + Schema + values constraints |
|---|---|---|---|
| TinySQL-125M (this model, mean of 3 seeds) | 4.5B | 77.95 ± 0.18 | 80.39 ± 0.20 |
| GPT-2 124M | not published | 78.93 | 82.36 |
| Pythia-160M | 300B | 60.78 | 67.63 |
| SmolLM2-135M | 2T | 82.57 ± 0.37 | 85.22 ± 0.21 |
| SmolLM2-360M | 4T | 87.03 ± 0.19 | 88.01 ± 0.12 |
The table reports execution accuracy (%): the generated query is run against the table and its result compared with the gold query's. Every model was fine-tuned with the same recipe after its own learning-rate sweep. This checkpoint is seed 0, which scores 78.01 greedy and 80.44 constrained. The official WikiSQL evaluator gives 78.08 execution and 72.38 logical-form accuracy for it.
model.generate() above is plain greedy decoding. The constrained numbers use the grammar-constrained decoder in the GitHub repository (decode/). It only allows the table's real column names and values copied from the question, which gives 100% valid SQL.
| Architecture | Llama-style decoder: 30 layers, hidden size 576, 9 query / 3 key-value heads (GQA), SwiGLU 1536, RoPE, RMSNorm, tied embeddings, context 2048 |
| Parameters | 125,077,824 |
| Tokenizer | own byte-level BPE, 32,768 tokens |
| Pretraining | 4.51B tokens (FineWeb-Edu, The Stack SQL, Stack Exchange, Gretel synthetic text-to-SQL), 8,600 steps, about 46 h on one NVIDIA L4 |
| Fine-tuning | WikiSQL train (56,355 examples), loss on the SQL only, 3 epochs, batch 32, peak LR 5e-5, best epoch by dev execution accuracy |
The architecture matches Llama, so the weights are distributed in the standard LlamaForCausalLM format. The model was built and trained with the repository's own code, not with transformers. The export is checked to produce the same token ids, logits and generated SQL as the original implementation.
AND conditions. No joins, GROUP BY, ORDER BY or nesting. Do not expect it to handle Spider-style or production SQL.FineWeb-Edu (ODC-By 1.0), The Stack (dedup), SQL subset (permissively licensed source files, subject to The Stack's terms of use), Stack Exchange dba and datascience (CC BY-SA; 2.5, 3.0 or 4.0 depending on the post date), gretelai/synthetic_text_to_sql (Apache-2.0), WikiSQL (Salesforce, BSD-3-Clause). The pretraining data was decontaminated against WikiSQL dev and test questions.
bibtex
@techreport{jha2026tinysql, title = {{TinySQL}: A 125M-Parameter Text-to-SQL Model Built and Pretrained From Scratch on 4.5B Tokens}, author = {Jha, Aditya Narayan}, year = {2026}, institution = {Zenodo}, doi = {10.5281/zenodo.23396770}, url = {https://doi.org/10.5281/zenodo.23396770}}from transformers import AutoModelForCausalLM, AutoTokenizer repo = "adiitya0929/TinySQL-125M-WikiSQL"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo) prompt = ( '<schema>CREATE TABLE t ("Player" TEXT, "No." TEXT, "Nationality" TEXT, "Position" TEXT, ' '"Years in Toronto" TEXT, "School/Club Team" TEXT);</schema>\n' "<question>What position does the player who played for butler cc (ks) play?</question>\n" "<sql>")ids = tok(prompt, return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=64) # greedy; stops at </sql>print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))# SELECT "Position" FROM t WHERE "School/Club Team" = 'butler cc (ks)';@techreport{jha2026tinysql, title = {{TinySQL}: A 125M-Parameter Text-to-SQL Model Built and Pretrained From Scratch on 4.5B Tokens}, author = {Jha, Aditya Narayan}, year = {2026}, institution = {Zenodo}, doi = {10.5281/zenodo.23396770}, url = {https://doi.org/10.5281/zenodo.23396770}}