GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
4thel00z
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
4thel00z
Model Tree
Qwen/Qwen3-VL-4B-Instruct
Input Modalities
Output Modalities
Supported Functionality
10,000 PDFs sampled from FinePDFs (Common Crawl), rendered with pdfboss at about 1224 px, labeled by kimi-k2.6, then cleaned:
Final: 60,315 page/JSON pairs, doc-disjoint eval split.
| Base | Qwen/Qwen3-VL-4B-Instruct, ViT and aligner frozen |
| Method | LoRA r=8, alpha=32, all-linear, lr 1e-4, bf16 |
| Schedule | 1 epoch, 3,770 steps, effective batch 16 (8x H100, DDP) |
| Context | max_length 6144, IMAGE_MAX_TOKEN_NUM 1024 |
| Framework | ms-swift 4.5.2 |
OmniDocBench (human-annotated, category-mapped, greedy IoU matching):
| English subset (755 pages) | v1 | v3 |
|---|---|---|
| Detection F1 at IoU >= 0.5 | 0.473 | 0.570 |
| Precision / Recall | 0.539 / 0.421 | 0.693 / 0.485 |
| Mean IoU of matched boxes | 0.603 | 0.691 |
| Class accuracy on matches | 90.6% | 91.2% |
| Valid JSON | 95.4% | 90.2% |
Internal held-out eval (1,652 pages, cleaned labels): F1 0.646 at IoU >= 0.5, mean IoU 0.710, type accuracy 93.3%, zero out-of-range boxes.
Tested with ms-swift:
bash
IMAGE_MAX_TOKEN_NUM=1024 swift infer \--adapters 4thel00z/kaktus-bbox-lora-v3 \--infer_backend pt --max_new_tokens 8192 --temperature 0
With transformers + peft:
python
from transformers import AutoModelForImageTextToText, AutoProcessorfrom peft import PeftModelbase = "Qwen/Qwen3-VL-4B-Instruct"model = AutoModelForImageTextToText.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(model, "4thel00z/kaktus-bbox-lora-v3")processor = AutoProcessor.from_pretrained(base)
Prompt with the page as the image and the instruction "Extract this page. Respond with only the JSON object." The system prompt is in args.json. Use max_new_tokens >= 8192 for dense pages and clip bbox values to [0, 1000].
TRAINING_HISTORY.md records the data cleaning steps, hardware, and the exact data snapshot. Training arguments are in args.json.