The hypothesis, and what worked
Can a compact language model inherit a robot policy's navigation behavior when its observations are expressed as text?
The experiment supports this hypothesis for the tested course and evaluator:
- Policy transfer produced useful navigation. On the same 20 selection seeds, the base model reached the target in 0/20 episodes; the 8-epoch adapter reached it in 20/20. After checkpoint selection was frozen, the adapter also completed 20/20 episodes on a separate reserved seed set. The PPO teacher completed 20/20 on those reserved seeds as well.
- A small supervised dataset was enough for this task. The model learned from 350 original training questions, balanced to 376 rows. Increasing the supervised budget from 2 to 8 epochs raised selection-course success from 10% to 100%, while validation question agreement rose from 43.1% to 100%. This comparison keeps the base model, data and other supervised settings fixed.
- Task success went beyond exact answer copying. The selected model agreed with the teacher on 100/108 held-out questions and still completed every reserved-seed navigation episode. These measure different abilities: matching individual decisions and reaching the goal through a sequence of decisions. Perfect teacher agreement was not required for successful navigation in these evaluations.
Why this release is useful: it connects an inspectable robot policy, a text decision interface, a trained adapter and a closed-loop evaluation. The matching inputs, exact question splits, alternative checkpoints and runnable Python recipe make that transfer available for inspection and reproduction.
PPO teacher → sensor-to-text questions → LoRA distillation → validation selection → reserved-seed evaluation
Results
Before and after adaptation

Table with columns: Policy, Test agreement, Selection success, Collision episodes, Success time| Policy | Test agreement | Selection success | Collision episodes | Success time |
|---|
| PPO teacher | reference | 100.0% | 0.0% | 7.45 s |
| Base model | 36.1% | 0.0% | 100.0% | — |
| LoRA · 2 epochs | 53.7% | 10.0% | 100.0% | 25.50 s |
| LoRA · 2 epochs + refinement | 50.9% | 0.0% | 55.0% | — |
| LoRA · 8 epochs (selected) | 92.6% | 100.0% | 0.0% | 7.60 s |
| LoRA · 8 epochs + refinement | 90.7% | 100.0% | 0.0% | 7.55 s |
How to read this comparison. Test agreement uses 108 questions; selection success uses 20 episodes with seeds 91001–91020, on course 7dc01cd0. Collision rate is the fraction of episodes with at least one collision and may overlap with successful episodes. Time is the mean for successful episodes. The checkpoint was selected by success, then validation agreement; test agreement was excluded from selection.
Reward refinement was tested from both supervised starting points. The 8-epoch supervised adapter and its refined version tied on selection success, while supervised validation agreement was higher (100% vs. 97.2%). The release therefore retains the supervised checkpoint. The comparison identifies what worked without assuming that an extra optimization stage must help.
Final evaluation on reserved seeds
The frozen checkpoint was evaluated on 92001–92020, separate from the selection episodes.
Table with columns: Measure, Selected Malibu-v1, PPO teacher| Measure | Selected Malibu-v1 | PPO teacher |
|---|
| Successful episodes | 20/20 | 20/20 |
| Episodes with collisions | 0.0% | 0.0% |
| Mean return | 23.237 | 23.227 |
| Mean successful-run time | 7.47 s | 7.40 s |

The selected policy matched the teacher's observed completion and collision rates in this evaluation. The small differences in time and return do not establish superiority.
What the model controls
- Goal:
goto_target — navigate to the target mug on course 7dc01cd0.
- Input: 21 observations rendered as Korean text: goal distance and bearing, motion, previous action and 11 distance sensors.
- Output: A = forward; B = veer left; C = veer right; D = turn left in place; E = turn right in place.
- Decision interval: 0.5 seconds. The evaluated rule scores the five option-token logits directly.
- Release format: a PEFT LoRA adapter for Qwen/Qwen3.5-0.8B; load the base separately.
- Controller role: high-level movement decisions. The fixed low-level walking controller is outside this adapter.
Use the model
Follow the Python setup and execution guide and install the compatible versions in requirements.txt. From a downloaded repository root:
from inference_helper import load_inputs, load_policyfrom connect_tycoon.link01 import questions as Q repo = "arabellako22/connectailab-rl-malibu-v1-link01"revision = "YOUR_REVIEWED_COMMIT" # Pin one model revision for reproducible inference.teacher, course = load_inputs(repo, revision=revision)policy = load_policy(repo, revision=revision, device="cuda")report = Q.drive_report(policy.pick, teacher, course, seeds=range(92001, 92021))print(report)
For one decision, policy.pick(Q.task_of(teacher.task, observation)) returns A, B, C, D or E. The released helper preserves the experiment's last-position vocabulary projection.
Training and compute
The run used NVIDIA A100-SXM4-80GB, bfloat16 and validation-based checkpoint selection. The matching teacher, course, observation/action contract and recorded reward weights were verified before data generation.
Table with columns: Supervised candidate, Balanced rows, Optimizer updates, Recorded wall time| Supervised candidate | Balanced rows | Optimizer updates | Recorded wall time |
|---|
| 2 epochs | 376 | 94 | 206.653 s |
| 8 epochs | 376 | 376 | 233.809 s |
These measured times include loading and evaluation and describe this dataset and runtime. Native logits_to_keep reduced vocabulary-projection work; decision-logit and LoRA-gradient smoke tests passed before training. See runtime_evidence.json for the original records.
Table with columns: Stage, Recorded setup| Stage | Recorded setup |
|---|
| Browser teacher | PPO; 20,480 decisions; 10 updates; seed 431; network 21 → 64 → 64 → 5 |
| Teacher optimizer | Learning rate 0.0003; gamma 0.97; GAE lambda 0.95; clip 0.2; 8 environments |
| Question splits | 350 train / 72 validation / 108 test, from disjoint episode seeds |
| Collection | 20% random actions; teacher labels; two matched-noise futures per action; up to 20 teacher decisions for action-value targets |
| Supervised candidates | 2 and 8 epochs from the same base; rank 8 / alpha 16; all linear layers; learning rate 1e-4; batch 8; seed 0 |
| Reward refinement | Two 100-step runs; learning rate 5e-6; batch 8; KL coefficient 0.2; up to 1,500 rows; seed 0 |
| Contract | link01-decision/goto_target/obs21-act5/walker51-14/v1 |
Table with columns: Reward term, Weight| Reward term | Weight |
|---|
| progress | +3 |
| reach | +20 |
| time | +0 |
| fall | -10 |
| out | -8 |
| collision | -4 |
The language-model refinement uses an offline, KL-regularized expected-advantage objective with simulated action scores. PPO trains the teacher. Rare-answer balancing produces the supervised rows above. Agreement after rounding numeric observations is 98.1%; this is a quantization diagnostic, not an accuracy bound.
Evaluation scope
The result demonstrates policy transfer on one fixed course, with separate selection and reserved episode seeds. Twenty final episodes give limited precision; performance on unseen layouts remains to be tested. The teacher was trained in MuJoCo, while decision-policy evaluation and the replay use a calibrated 2D walking approximation without falls or the low-level walking network. Physical-robot performance was not evaluated.
Files and reproduction
adapter_model.safetensors, adapter_config.json: the selected adapter.
brain.json, world.json, input_manifest.json: matching teacher, course and input hashes.
results.json, selection.json, heldout.json, teacher_heldout.json: policy comparison, selection rule and reserved-seed results.
dataset/, alternative/: exact question splits and alternative trained adapters.
source/run_experiment.py, source/experiment_config.json, source/README.md: execution code, active settings and exact commands.
replay.mp4, assets/replay_poster.png, assets/replay_trace.json, source/replay.py: a reproducible fixed-seed comparison, video and provenance.
source/publish_public_release.py, source/check_public_release.py: guarded publication and privacy checks.
environment.json, requirements.txt, experiment_meta.json, runtime_evidence.json, inference_helper.py: environment, measured budgets and inference implementation.
release_verification.json, SHA256SUMS: artifact provenance and file integrity.
Credits and license
Built with CONNECT TYCOON, CONNECT AI LAB, AI CITY BUILDERS and Qwen. Framework snapshots retain their original attribution. Apache-2.0; the base model retains its own license and attribution requirements.