GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
4thel00z
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
4thel00z
Model Tree
Qwen/Qwen3-VL-4B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Distilled from kimi-k2.6: the teacher extracted layout JSON for 10,000 PDFs sampled from FinePDFs (Common Crawl), rendered with pdfboss at about 1224 px. The student trained on 60,284 page/JSON pairs with a doc-disjoint eval split.
| Base | Qwen/Qwen3-VL-4B-Instruct, ViT and aligner frozen |
| Method | LoRA r=8, alpha=32, all-linear, lr 1e-4, bf16 |
| Schedule | 1 epoch, 3,768 steps, effective batch 16 (2x RTX 4090, DDP) |
| Context | max_length 6144, IMAGE_MAX_TOKEN_NUM 1024 |
| Framework | ms-swift 4.5.2 |
| Final losses | train 0.115, eval 0.134 |
Generation eval on 1,652 held-out pages (greedy, max_new_tokens 4096):
| Metric | Value |
|---|---|
| Valid JSON | 98.0% (all failures are max-token truncations on dense pages) |
| Element count | 13,558 predicted vs 13,532 reference; exact per-page match 65.8% |
| Detection P / R / F1 at IoU >= 0.5 | 0.614 / 0.616 / 0.615 |
| Mean IoU of matched boxes | 0.675 |
| Type accuracy on matched boxes | 93.0% |
| Out-of-range bboxes | 0.21% |
Reference labels come from the teacher model.
Tested with ms-swift:
bash
IMAGE_MAX_TOKEN_NUM=1024 swift infer \--adapters 4thel00z/kaktus-bbox-lora-v1 \--infer_backend pt --max_new_tokens 8192 --temperature 0
With transformers + peft:
python
from transformers import AutoModelForImageTextToText, AutoProcessorfrom peft import PeftModelbase = "Qwen/Qwen3-VL-4B-Instruct"model = AutoModelForImageTextToText.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(model, "4thel00z/kaktus-bbox-lora-v1")processor = AutoProcessor.from_pretrained(base)
Prompt with the page as the image and the instruction "Extract this page. Respond with only the JSON object." The system prompt is in args.json. Use max_new_tokens >= 8192 for dense pages and clip bbox values to [0, 1000].
TRAINING_HISTORY.md records hardware moves, config changes mid-run, and the exact data snapshot. Training arguments are in args.json.