TL;DR
Table with columns: Metric (100 val QAs, seed=42), v1 (154k, no speed), v3-20k (this, speed), v3 (154k, speed)| Metric (100 val QAs, seed=42) | v1 (154k, no speed) | v3-20k (this, speed) | v3 (154k, speed) |
|---|
| Exact | 62 | 64 | 66 |
| Perception | 16 | 13 | 16 |
| Prediction | 22 | 22 | 22 |
| Planning | 13 | 12 | 13 |
| Behavior | 11 (44%) | 17 (68%) ⭐ | 15 (60%) |
Pick this variant when:
- You care about behavior classification specifically (68% vs v3's 60%).
- You want an existence proof that speed grounding + a modest data budget beats a much larger visual-only run.
- You're re-training on a new dataset and want a fast sanity checkpoint before committing to full-scale.
Pick v3-full when:
- You want the highest total accuracy across all levels (66 vs 64).
- Perception matters — v3-full is 16 vs this variant's 13.
How to use
Identical to v3 — see the v3 README for the full example. Just swap the adapter path to this repo.
Training details
- 20,000 QAs (uniformly random from the 154 k train pool,
seed=42).
- 1 epoch, 2 h 40 min on 1× L40S 48 GB (7.4× less compute than v3-full's 19h50min).
- Same LoRA config (r=32, α=64, target = 7 modules), same
lr=2e-5, cosine schedule, bsz=1, grad-accum=8, bf16.
eval_loss plateau 0.184 (v3-full 0.175, essentially matched).
Why is behavior higher on the smaller run?
Speculation, not proven: with less data the model over-relies on the numeric speed prefix (which perfectly predicts the "fast/slow/…" bin) and under-fits the tiny visual variations that push it to output non-canonical answers. At 154k the model sees enough perception / prediction / planning examples to develop a more "distributed" style that occasionally deviates from the terse behavior template.
Practical takeaway: if you're deploying, keep both variants around and pick per-QA-level, or use v3-full which is the safer overall bet. This variant is included for reproducibility of the finding.
Everything else (base model, dataset, license, limitations, citation) — see v3.