GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
evalstate
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
evalstate
Model Tree
Qwen/Qwen2.5-VL-3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Qwen/Qwen2.5-VL-3B-Instruct (the paper's own SFT base).
4-bit NFQ (bitsandbytes, double-quant, bf16 compute) · LoRA r=16/α=16, dropout 0 ·
targets q,k,v,o,gate,up,down_proj · paged_adamw_8bit · lr 2e-4 linear · effective batch 4 ·
wd 0.01 · seed 3407 · gradient checkpointing on · 200-image train subset · 1 epoch (2 planned;
epoch 2 was cut by a host-RAM limit on a10g-small — see sft_results.json).
| Epoch | Train loss | Subset eval (100 img) |
|---|---|---|
| 1 (best) | 0.163 | 52.0% |
| Run | Daylight-phase acc |
|---|---|
| Narrow zero-shot (this base, no adapter) | 21.3% |
| Narrow fine-tuned (this adapter) | 56.0% |
Fine-tune delta: +34.7 points. The adapter more than doubles daylight-phase accuracy, though the model collapses to the majority class (Afternoon) rather than learning physically grounded daylight reasoning — see the report's failure analysis. Full metrics, predictions, and the deterministic scorer are in evalstate/timespot-daylight-eval.
python
import torchfrom transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessorfrom peft import PeftModelbase = Qwen2_5_VLForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct", torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "evalstate/Qwen2.5-VL-3B-TimeSpot-daylight-LoRA")processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")
Training script: code/train_sft.py.
Eval + scoring scripts are in the same dataset repo. Part of a reproducible TimeSpot evaluation
(see the accompanying report for protocol-sensitivity analysis).