GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
paawandesai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
paawandesai
Model Tree
Qwen/Qwen3-4B
Input Modalities
Output Modalities
Supported Functionality
Exact files (checked by code), with mean Haiku judge scores (1-5) as spec adherence / robustness. Eval sets are in the repo (data/ach/), seeds disjoint from training.
| Model | eval (50) | holdout (100) | bigger files, unseen names (50) |
|---|---|---|---|
| Qwen3-4B base | 0/50 (1.04 / 1.06) | - | - |
| This adapter (3,000 files, 1 epoch) | 38/50 (4.56 / 2.96) | 85/100 (4.71 / 2.83) | 0/50 (1.42 / 1.46) |
Robustness is held down by the judge counting the worksheet before the file as commentary; pass/fail is decided on the file alone.
Frontier ceiling, 30 specs (seeds 400-429), best prompt per model: Claude Opus 5.5 26/30, GPT-5.5 6/30, Claude Sonnet 5.5 2/30. On the first 20 of those specs this adapter gets 15/20 (99% of lines correct) vs Opus 17/20, GPT-5.5 3/20, Sonnet 1/20.
Known limit: on bigger files with names it never saw it gets 0/50 exact, because it learned the padding of its ~180 training names rather than padding to field width in general.
bash
python eval.py --model paawandesai/qwen3-4b-nacha-writer --eval-set data/ach/holdout.jsonl --engine vllmOne command runs this adapter and the untuned base on the same prompts, checks every file in code, runs the Haiku judge (with ANTHROPIC_API_KEY set) and prints the base-vs-tuned table. Pinned eval code: repo commit dad58ad.
Code, eval sets and training script: https://labs.gauntletai.com/paawandesai/moe_slm (infra/ach_train.py, eval.py, finspread/ach.py).
Training data is synthetic, generated by code from seeds 100000+; eval sets use disjoint seeds.
Training: LoRA r32/alpha64 on all projections, lr 2e-4 cosine, batch 8, max length 12,288, bf16, TRL SFTTrainer.