Training
Each step, per GPU: 16 episodes (an original question, a silently edited evidence state, or an announced revision, mixed 0.4 / 0.35 /
0.25); 4 replies sampled per episode at temperature 1; each reply scored by the rule engine; advantage = reward minus the mean reward of
that episode's samples; with probability 0.5 the reference reply is added as one more candidate with advantage 1 minus the mean sampled
reward. One policy-gradient step on the candidates with the largest absolute advantage.
Table | |
|---|
| Base model | Qwen/Qwen3.5-9B, frozen |
| Adapter | LoRA r = 32, alpha = 64, no dropout, on every projection of the language model (q/k/v/o, Gated DeltaNet in/out projections, MLP); 86.6 M trainable parameters. The vision tower and the vision merger are not adapted. |
| Data | train split of the EcoWitness v10 data, English: 80,000 episodes drawn from 274,181 conversations |
| Reward | answer 0.6 (graded partial credit up to 0.5 for near answers), load_bearing F1 0.25, support F1 0.15, changed line 0.15 on revisions |
| Schedule | 2,500 steps, 1,250 updates of 64 episodes (2 GPUs x 16 episodes x gradient accumulation 2); AdamW, lr 5e-5, 3 % warm-up, cosine decay to 10 %; bf16; seed 0 |
| Hardware and time | 2 x NVIDIA H100 80 GB, 17.5 hours; peak memory 40 GB per GPU |
Compared with configs/rl.yaml (written for 4 GPUs x 8 episodes), this run used 2 GPUs x 16 episodes with both token budgets doubled
(gen_token_budget 64,000, train_token_budget 16,000) and no time limit; steps, updates and episodes per update are the same. The exact
configuration is in training/config.json, the launch script in training/run_rl_lambda.sh.
Learning rate, loss and sampling statistics every 10 steps are in training/training_log.csv (plotted in
training/lr_loss_curves.png); the learning rate peaks at 5e-5 at step 80 and ends at 5e-6. The loss is the policy-gradient surrogate
(advantage-weighted token log-likelihood), which stays near zero and shrinks as the sampled replies agree more; it is not a fit error.
checkpoints/checkpoint-2250 and checkpoints/checkpoint-2500 hold the adapter with the optimiser and scheduler state
(state.pt), for continuing the run with --resume true; checkpoint-2500 has the same weights as the adapter at the top level.
The full record of the run (terminal logs, environment, per-window tables) is in results/lambda_rl_20261112/ of the code repository.
Validation during training
Greedy replies on a fixed sample of 240 validation records (80 per record type); sites in train, validation and test do not overlap.
correct is answer accuracy; the 95 % interval of an accuracy near 0.9 on 240 records is about +/- 4 points.
Table with columns: step, reward, correct, original question, silent edit, announced revision, load_bearing F1, support F1| step | reward | correct | original question | silent edit | announced revision | load_bearing F1 | support F1 |
|---|
| 0 | 0.608 | 0.608 | 0.600 | 0.562 | 0.662 | 0.536 | 0.541 |
| 250 | 0.888 | 0.804 | 0.775 | 0.725 | 0.912 | 0.945 | 0.985 |
| 500 | 0.931 | 0.875 | 0.887 | 0.800 | 0.938 | 0.976 | 0.996 |
| 750 | 0.937 | 0.883 | 0.850 | 0.850 | 0.950 | 0.974 | 0.992 |
| 1000 | 0.940 | 0.892 | 0.875 | 0.825 | 0.975 | 0.977 | 0.993 |
| 1250 | 0.947 | 0.904 | 0.900 | 0.887 | 0.925 | 0.980 | 0.997 |
| 1500 | 0.948 | 0.908 | 0.875 | 0.887 | 0.963 | 0.972 | 0.998 |
| 1750 | 0.963 | 0.938 | 0.925 | 0.912 | 0.975 | 0.980 | 0.992 |
| 2000 | 0.962 | 0.929 | 0.925 | 0.875 | 0.988 | 0.983 | 0.996 |
| 2250 | 0.964 | 0.938 | 0.938 | 0.887 | 0.988 | 0.982 | 0.993 |
| 2500 | 0.963 | 0.938 | 0.925 | 0.900 | 0.988 | 0.974 | 0.994 |
Accuracy by question group, before training and at the end (a few to a few dozen records per group, so single groups are noisy):
Table with columns: group, step 0, step 2,500| group | step 0 | step 2,500 |
|---|
| A image | 0.55 | 0.70 |
| B community traits | 0.49 | 0.95 |
| C fusion | 0.33 | 0.81 |
| D diversity | 0.70 | 0.97 |
| E river network | 0.67 | 1.00 |
| K species knowledge | 0.67 | 1.00 |
The image questions (group A) gain the least; the adapter does not touch the vision side. Training curves: training/curves.png;
per-step logs: training/rl_log.jsonl, training/val_log.jsonl.
Test set
7,523 questions from sites that appear in neither training nor validation, 73,245 conversations; greedy decoding. All metrics:
eval_test/metrics.json (definitions in ecowitness/score.py of the code repository).
Table with columns: record type, records, answer accuracy (%), 95 % interval| record type | records | answer accuracy (%) | 95 % interval |
|---|
| original question | 7,523 | 90.9 | [90.2, 91.5] |
| silent counterfactual | 32,461 | 91.0 | [90.8, 91.4] |
| announced revision | 25,738 | 95.7 | [95.5, 96.0] |
| revision chain, last step | 7,523 | 81.0 | [80.1, 81.9] |
By question group (%):
Table with columns: group, original question, silent counterfactual, announced revision| group | original question | silent counterfactual | announced revision |
|---|
| A image | 72.3 | 75.9 | 65.8 |
| B community traits | 95.0 | 93.7 | 98.8 |
| C fusion | 92.6 | 88.0 | 89.5 |
| D diversity | 94.1 | 92.4 | 98.4 |
| E river network | 100.0 | 99.7 | 99.6 |
| K species knowledge | 98.4 | 96.9 | 99.3 |
Silent counterfactuals (the edited evidence asked as a new question, without notice):
Table with columns: metric, value (%)| metric | value (%) |
|---|
| invariance rate (answer unchanged when the truth is unchanged) | 95.2 [94.8, 95.5] |
| flip rate (answer changed when the truth changed) | 90.1 |
| flip accuracy (changed to the new correct answer) | 89.0 [88.5, 89.6] |
| permutation equivariance | 95.8 |
| attribution faithfulness (stated load-bearing witness vs whether the answer changes when it is removed; agreement) | 95.3 (F1 95.1, n = 10,419) |
| notice dependence (flip accuracy with notice minus silent flip accuracy) | +4.2 |
Table with columns: edit axis, records, invariance rate, flip accuracy| edit axis | records | invariance rate | flip accuracy |
|---|
| taxonomy | 6,481 | 99.0 | 84.0 |
| trait | 1,264 | 99.9 | 93.9 |
| abundance | 918 | 98.1 | 89.7 |
| status | 1,085 | 96.6 | 91.1 |
| witness | 10,470 | 98.2 | 92.0 |
| image | 3,433 | 69.4 | 81.6 |
| record | 1,287 | 99.6 | 99.0 |
Announced single-step revisions:
Table with columns: metric, value (%)| metric | value (%) |
|---|
| accuracy when the answer must change (flip) | 93.1 |
| accuracy when it must stay (keep) | 97.7 |
| inertia rate (should change, did not) | 2.0 [1.8, 2.2] |
| contagion rate (changed after an unrelated edit) | 0.18 |
| redundancy over-reaction rate (changed after an edit to a cited but redundant witness) | 5.8 |
changed F1 | 97.4 |
Revision chains (2 to 5 steps; each step sees the model's own earlier replies):
Table with columns: step, 0 (question), 1, 2, 3, 4, 5| step | 0 (question) | 1 | 2 | 3 | 4 | 5 |
|---|
| accuracy (%) | 90.9 | 88.7 | 85.7 | 82.4 | 80.5 | 77.0 |
Whole chain correct 69.7 %; drift (accuracy on the question minus accuracy at the last step) 9.9 points; recovery after an error 38.5 %; slip after a correct step 8.8 %.
Citations (original questions): support F1 99.8, load_bearing F1 98.3, correct answer with the exact load-bearing set 88.5 %, no invented identifiers. Every reply parses.
Question families with the lowest accuracy on original questions: A5-Sinuosity 45.2, A6-RiparianVegetation 48.2, B6-StatusCount 60.1, A7-BuiltUpBank 64.0, C1-SceneAndCommunity 64.8, A1-NearshoreCover 71.4.
Reading the numbers:
- Images are the weak point. Group A reaches 72.3 % on original questions, revisions that swap the image 68.4 %, and the invariance rate under silent image edits is 69.4 %. The adapter does not touch the vision side.
- The held-out family C1 (not in training) reaches 64.8 % on original questions, well below the trained families; it also depends on the image.
- Accuracy falls along a chain by 2 to 3 points per step, and only 38.5 % of errors are repaired later. Training used single-step records only.
- No zero-shot test result of the base model was run, so the gain over the base model on the test set is not measured here; on the 240 validation records it was 60.8 % before training. Synthetic references: random choice 25.3 % on original questions; "never change" 55.7 % on revisions with an inertia rate of 100 %.
- One training run, one seed; the intervals are bootstrap intervals over test records and do not include training randomness.
Use
import torchfrom peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoProcessor base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.5-9B", dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "ForeverBlue/EcoWitness-Qwen3.5-9B-RL-LoRA").merge_and_unload()processor = AutoProcessor.from_pretrained("ForeverBlue/EcoWitness-Qwen3.5-9B-RL-LoRA")
Needs transformers >= 5.3 and peft >= 0.18. The adapter was trained with the chat template's reasoning block closed
(enable_thinking=False) and expects prompts in the witness-protocol format; ecowitness/protocol.py renders them and
python -m ecowitness.evaluate --model Qwen/Qwen3.5-9B --adapter <local copy of this repository> runs the full evaluation.
Limits
- Trained and validated only on this benchmark's question families and evidence format; not a general ecology assistant.
- English only.
- One training run, one seed.