Method
Round 4 rebuilds the round-1 recipe (VK-OPD-think, "ck80") and changes one thing per arm:
B0 = the round-1 recipe with a new seed; B1 = B0 with a blended knowledge teacher; B2 = B1 with premise rows on the photo side.
Teachers are the frozen base model with privileged context; advantage per token = log π_teacher − log π_student on the student's own samples
(sampled-token reverse KL through TRL's GRPO loss, no reward, beta 0). The student's user turn (photo, question, any premise) is the same for student and teacher.
- Photo questions (V teacher, all arms): photo + system "Answer from what is actually shown in the image, even if it differs from what is typical."
- Knowledge questions (K teacher): a blend: per-token advantage 0.5 · A_K0 + 0.5 · A_KI, where K0 = the base model without the photo and KI = the base model with the photo and the system line "Answer with what is generally true in the real world, even if the photo shows something different."
- Data: the private synthetic training pool (counterfactual and normal photos with paired knowledge and photo questions); knowledge items kept only if the no-photo teacher answers them correctly in 2 of 2 samples (CK 548, NK 130 items); photo items ungated (CV 1,995, NV 246). Plus premise rows: the same photo questions with a typical-fact sentence in front (CF on counterfactual photos, NF on normal photos; the labmate's primed training items), so the student learns to answer from the photo when the text states the typical fact.
- Per step: 6 CK, 2 NK, 4 CV, 2 CF, 1 NV, 1 NF prompts (static mix).
- 120 steps, lr 3e-6 constant after 5 warmup steps, AdamW, grad clip 1.0, 16 prompts × 4 rollouts per step, rollouts thinking on, temperature 1.0,
up to 4,096 tokens; fp32 master weights, bf16 compute (FSDP2 with CPU offload), 4× H100 (bioai job 338693).
Interim evaluation
Private RealViK-nat rc2 benchmark, interim: 741 pairs (the fixed Table 3 set K* ∩ rows unchanged since the 11-07 release), grader v5 with
Qwen3.5-27B (FP8) as judge through OpenRouter, thinking on, card sampling (T 1.0, top-p 0.95, top-k 20, presence 1.5), 8,192 tokens, 1 sample.
These numbers will be replaced by the full table (1,054 pairs, self-served judge).
Conditions: CV / CK = counterfactual photo, answer from the photo / from world knowledge; NV / NK = the same on a normal photo; CF = CV with a
typical-fact sentence in front; CI = CK with an instruction to inspect the photo; KT = the knowledge question without a photo.
C-Joint = CV and CK both correct; All-Joint = CV, CK, NV and NK all correct.
Table with columns: model, CV, CK, NV, NK, CF, CI, KT, C-Joint, All-Joint| model | CV | CK | NV | NK | CF | CI | KT | C-Joint | All-Joint |
|---|
| Qwen3.5-9B (thinking) | 82.5 | 71.1 | 93.5 | 95.4 | 86.4 | 67.3 | 97.3 | 56.1 | 52.0 |
| round 1 (VK-OPD-think) | 85.3 | 97.0 | 93.3 | 97.6 | 81.2 | 93.0 | 96.8 | 82.6 | 76.7 |
| this model | 82.9 | 90.4 | 94.3 | 97.2 | 85.6 | 79.6 | 97.2 | 74.6 | 70.3 |
C-Joint vs the base: +18.5 [+14.6, +22.4] pp; vs round 1: −8.0 [−11.1, −5.0] pp (paired pair bootstrap 95% CI, 2,000 resamples). One seed.
Shared privately for research comparison; not a release.