GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ChetanDhembreAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ChetanDhembreAI
Model Tree
Qwen/Qwen3-4B-Instruct-2507
Input Modalities
Output Modalities
Supported Functionality
Full 2,000-row test set of LiquidAI/ifstruct-v1.0,
scored with the benchmark's own validator (byte-identical to Liquid4All/ifstruct).
| Model | pass@1 | JSON | YAML |
|---|---|---|---|
| this model (merged) | 95.75 | 95.2 | 96.3 |
| same LoRA served un-merged | 96.45 | 96.6 | 96.3 |
| Qwen3-4B-Instruct-2507 (base; earlier run, max_tokens 4096) | 76.15 | 76.4 | 75.9 |
Eval setup: greedy (temperature 0), 1 sample per prompt, max_tokens 16000, no system prompt, the model's chat
template (non-thinking model), no constrained decoding, vLLM 0.25. Per-row outputs (prompt, response, validator
errors) are in eval/. The 0.7-point gap to the un-merged adapter comes from bf16 rounding of very small
LoRA deltas during the merge; it is within greedy run-to-run variation (2.4% of rows flip).
Remaining failures (85/2000) are concentrated in prompts with long, heavily escaped free-text payloads (email threads, terminal sessions, dialogue); 17 are degenerate repetition loops inside a string value.
This is not a blind held-out score. (1) The synthetic data generator was steered toward the failure types the model showed on this benchmark (no benchmark rows were copied). (2) Checkpoints were compared on validation slices drawn from this benchmark (≈288 rows) and on full-benchmark evaluations. Expect a lower number on genuinely unseen structured-output distributions.
python
from transformers import AutoModelForCausalLM, AutoTokenizername = "ichetandhembre/Qwen3-4B-Instruct-2507-ifstruct"tok = AutoTokenizer.from_pretrained(name)model = AutoModelForCausalLM.from_pretrained(name, torch_dtype="auto", device_map="auto")Or vllm serve ichetandhembre/Qwen3-4B-Instruct-2507-ifstruct.