Usage notes
Serve with chat_template_kwargs={"enable_thinking": True}. The Qwen3.5-0.8B chat template turns thinking OFF by default, but every training target begins with a <think> block. The base model's multi-token-prediction (mtp.*) weights are not included, so MTP speculative decoding is unavailable; ordinary serving is unaffected.
Training
- Data: OpenWebRL SFT trajectories, 3,084 per-turn rows, rendered with LlamaFactory's
qwen3_5 template (history masked; only the final assistant turn of each row is trained).
- Full fine-tune with LlamaFactory: vision tower frozen (88,004,352 params); projector and language model trained (764,981,568 of 852,985,920 params).
- 3 epochs (75 optimizer steps), learning rate 1e-5, cosine schedule, warmup 0.1, effective batch 128 (1 per device x 64 accumulation x 2 GPUs), bf16, DeepSpeed ZeRO-2, cutoff 36,864 tokens, image_max_pixels 12,845,056.
- 2 x H200, about 45 minutes.
- Mean training loss by epoch: 0.829 / 0.591 / 0.479. No held-out split was used; most of the drop after epoch 1 comes at epoch boundaries (repeat passes over the data).
Evaluation
Not yet evaluated.