GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
isaacmg
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
isaacmg
Model Tree
Qwen/Qwen3-VL-8B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Two findings survive rigorous noise control; per-page claims do not (see methodology note).
Eval loss (teacher-forced, sampling-free): best in the series at every step — 0.5444 at 1200, 0.5310 at 1800, roughly 0.028 below both the merger-LoRA twin and the frozen-merger flagship at their step-1300 finals. The only loss-curve separation in the series, and it kept widening (decelerating) through 1800.
Transcription benchmark medians: statistically tied. Religious manuscripts 0.811 (flagship 0.816, merger-LoRA 0.810); frozen PGP-131 0.870 (0.862 / 0.868). The likelihood gain never translated into transcription-benchmark gains — at 1200 or at 1800.
Zero-shot grounding (none of these arms saw grounding data): merger training visibly reorganizes spatial association. On a frozen 133-query grounding eval (phrase→box, box→text, grounded page transcription):
| arm | locate median IoU | read_box CER | grounded line-box IoU |
|---|---|---|---|
| v1.9a-1300 (frozen merger) | 0.288 | 0.579 | 0.150 |
| v1.9b-1300 (merger LoRA) | 0.278 | 0.543 | 0.437 |
| v1.9c-1200 (full) | 0.277 | 0.820 | 0.253 |
| v1.9c-1800 (full, +600 steps) | 0.318 | 0.771 | 0.355 |
Both trained-merger arms emit far better-localized line boxes than the frozen arm (line IoU 0.437 / 0.355 vs 0.150) while transcription medians stay tied — and the effect is undirected: full-weight training initially degrades region-conditioned reading (read_box 0.820 vs 0.579), then partially recovers while every spatial metric improves monotonically through step 1800. The merger is still actively reorganizing spatial pathways late in training, from transcription-only supervision.
Methodology note. Per-page "break/rescue" comparisons between these arms are unreliable at single-decode granularity: a same-model rerun control (identical weights, temperature 0.1) flipped 4 of 29 probed fragments across the 0.5 aligned-F1 line, including the largest swings previously attributed to merger effects. A tail of fragments is bistable under sampled decoding; page-level attribution requires N-decode averaging or greedy evaluation. Medians, loss curves, and the aggregate grounding rates above are robust to this and are the basis of every conclusion here.
Conclusion: full-weight merger training improves likelihood but not transcription benchmarks, so v2.0a keeps the merger frozen (simpler, equal). But the grounding probe shows the merger is precisely where prompt↔visual-region association lives — motivating a v2.0b arm that re-enables merger training on v2.0's grounding-supervised mixture, judged on grounding metrics with transcription medians as the no-regression guard. Single-seed caveat applies throughout.
One line per generation — which checkpoint to use and which are historical:
| version | repo | status | canonical revision | headline (benchmark) |
|---|---|---|---|---|
| v2.0a | v20a-ckpt | current best on both corpora; first with grounding | step 1800 af9df6a0 | F1 0.849 (religious-140) / 0.875 (PGP-131); locate IoU 0.467, read_box CER 0.129 |
| v1.9a | v19a-ckpt | FLAGSHIP — use this | step 1300 |
* CER over substantive attempts only, alignment-based scorer. Benchmarks differ across generations (Talmud print vs Genizah manuscripts) — compare within a row's named benchmark, not across rows.
43e21bd7| F1 0.816 / CER* 0.216 (Genizah religious-140); F1 0.862 / CER* 0.196 (frozen PGP-131) |
| v1.9c | v19c-ckpt | experimental — not adopted | step 1800 6620fb0d | full-weight merger study; best eval loss (0.531); medians tied; strongest zero-shot grounding drift |
| v1.9b | v19b-ckpt | experimental control — not adopted | step 1300 f8618c28 | merger-LoRA ablation study; loss ≡ v1.9a; best zero-shot grounded line boxes (IoU 0.437) |
| v1.8b | v18b-ckpt | superseded; warm-start ancestor of all v1.9 | step 700 c80313f8 | first arm with a live vision-tower LoRA |
| v1.8a | v18a-ckpt | superseded A/B control (vision frozen) | step 700 | control arm for the v1.8 vision experiment |
| v1.7 | v17-ckpt | superseded | step 800 | first Genizah-handwriting generation; best VLM on both corpora at its era's benchmarks (Aug 2026) |
| v1.6 | v16-ckpt | superseded; Talmud-print reference | step 1000 (official); step 1100 b9f47f32 (v1.7 warm start) | Talmud page CER: gemara 0.090 / rashi 0.047 / tosafot 0.099 — Rashi-script 6.7× better than the best closed model we tested (0.315) |
| v1.5 | rashi-ckpt | superseded | final e050aca8 (step 2000) | proved pure Rashi-glyph perception: CER 0.018 on unmemorizable synthetic text |
| v1 | hebrew-ckpt | superseded (language-only LoRA) | step 3800 e2f85dde | first checkpoints to read Vilna gemara from pixels |