Results
Solved episodes on held-out instances.
Table with columns: Level, What it asks, Base, This adapter| Level | What it asks | Base | This adapter |
|---|
| N0 | warm-up, method given | 28/180 | 17/24 |
| N1 | warm-up, method not given | 0/60 | 18/24 |
| N2 | foreign load unit | 0/9 | 0/24 |
| N3 | cracked collar | 0/9 | 2/24 |
| N4 | soaked against dry tests | 0/9 | 15/24 |
- The base model's N0 and N1 cells predate the ladder recalibration.
- N0 and N1 hold no decision, and a fixed script solves them.
- Both N3 hits are control instances.
- N4 is not a learned decision: the demonstrations contained the rain filter.
- This is one training seed. The 24 episodes are 8 tasks with 3 attempts each.
Training
- Data: 1,200 oracle demonstrations, each graded and produced with the real tools. Outputs from closed models were never training data.
- Method: SFT, 200 steps, about 20 minutes on one RTX 5090.
- Adapter: LoRA rank 16, alpha 32, on the attention and MLP projections.
- Stack: prime-rl and verifiers, with vLLM for evaluation.
Use
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = "Qwen/Qwen3-4B-Instruct-2507"tokenizer = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "arthurcs01/geotech-rl-qwen3-4b-sft")
The results above came from vLLM, not PEFT. This recipe was checked on the real architecture cut to two layers with random weights, not on the full model.
What differs from the training run
The tensors are unchanged. Their names were converted to PEFT's format, because prime-rl saves a different one. With the original names PEFT loads the file, warns about missing keys and applies nothing.
License and citation
Apache 2.0, as the base model. These weights modify Qwen3-4B-Instruct-2507 by the Qwen team.
To cite the work, use the code's DOI: 10.5281/zenodo.23861866. The repository describes how AI was used in the project.