GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
caiotheodoro
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
caiotheodoro
Model Tree
Qwen/Qwen3-VL-8B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Seed-777, n=1000, 739 FLAG, zero signature overlap with train. Eval re-renders pages from structured fields.
| System | Recall | HIGH | Prec | Parse |
|---|---|---|---|---|
this adapter (sft-dedhi) | 0.959 | 0.969 | 0.956 | 1.0 |
prior published (sft-limithi) | 0.839 | 0.893 | 0.870 | 1.0 |
| GPT-5.6 Luna, zero-shot vision | 0.373 | 0.388 | 0.344 | 0.972 |
| Qwen3-VL-8B, no adapter | 0.098 | 0.115 | 0.278 | 0.973 |
The written contract is >0.95 recall, >0.95 precision, 1.00 HIGH. Rec and prec hold. HIGH is 0.969. DEDUCTIBLE_HIGHER 0.957 / 0.846 (was 0.507 / 0.407). Citation exact-match ≈ 1.0 is gold-string cloning; type recall is the money metric. Full tables in the repo docs/BENCHMARK.md.
Issuance QC is a closed diff over a finite ISO/ACORD-style ontology (named insured, period, coverages, limits, deductibles, form numbers, additional insureds, premium). Carriers still run it with people and Guidewire / Duck Creek rules. There is no public VL benchmark for binder-vs-policy consistency. Nearby datasets (FUNSD, CORD, DocVQA, VRDU) extract fields; they do not ask whether two documents agree. Claims NLP is a different problem (fraud, coverage opinion) and needs a different oracle.
A seeded generator plus a deterministic verifier (suture_forge.verify) produces exact gold. The same recipe is what let a 1.7B LoRA beat a frontier model on ReconForge, a different closed task. Zero-shot general VL does not know this taxonomy — that is the 0.098 / 0.373 rows above.
python
from peft import PeftModelfrom transformers import AutoProcessor, Qwen3VLForConditionalGenerationbase_id = "Qwen/Qwen3-VL-8B-Instruct"model = Qwen3VLForConditionalGeneration.from_pretrained(base_id, device_map="auto")model = PeftModel.from_pretrained(model, "caiotheodoro/suture-8b")processor = AutoProcessor.from_pretrained(base_id)
Two images: binder page, then policy page. Output schema is in the repo CONTRACTS.md. Training and eval in this project used Modal 4-bit, 384px (cloud/modal_eval.py).
Sample pages (forge-rendered, not carrier scans) under samples/: PASS pair, DEDUCTIBLE_HIGHER pair, AI_MISSING pair.
Synthetic only. Seeds: 7 (train/val), 11, 13, 17, 19, 23 (class-conditional mixes). Seed 777 is eval-only. JSONL is on the dataset repo, not in git.
Chain: sft-vl → sft-distill → sft-restem → sft-numeric → sft-limithi → sft-ded → sft-dedhi. Continue-from-adapter SFT, 1 epoch per mix, Modal L4. This adapter: 1280 steps from sft-ded. This Hub repo is PEFT only (~200MB). No merged 16-bit 8B, no train JSONL here.
AI_MISSING 0.912, FORM_MISSING 0.944.bibtex
@software{theodoro2026suture,author = {Theodoro, Caio},title = {Suture: vision-language policy-issuance QC with a verifier-as-oracle},year = {2026},url = {https://github.com/caiotheodoro/suture},note = {seed-777 n=1000: recall 0.959, HIGH 0.969, precision 0.956, parse 1.0}}