HuggingEnvs

geoguesser-qwen3.5-4b-grpo

README

License: apache-2.0

What it scores

200 held-out tasks across 73 countries, pass@4, 800 episodes per arm, all scored through the same curve. The full board is in the project README.

Table with columns: model, mean-of-4, best-of-4, median error
model	mean-of-4	best-of-4	median error
claude-sonnet-5	0.6952	0.8397	324 km
this adapter (Qwen3.5-4B + LoRA)	0.6445	0.7092	662 km
gpt-5.4-mini	0.5732	0.7719	753 km
claude-haiku-4.5	0.5374	0.7014	939 km
Qwen3.5-122B-A10B	0.5338	0.6987	767 km
Qwen3.5-4B, untrained	0.4825	0.6589	1226 km

Against its own base, paired per task: +0.1620 ± 0.0137 (95% CI), better on 169 of 200 tasks. Country identified 21.2% → 33.1%. Episodes floored at zero on the environment's own curve 29.5% → 0.5%.

Scores are on the training reward curve, min(1, 0.5·e^(-d/1492.7) + 0.5·e^(-d/5000)) × (1 - min(cost, 0.2)), recomputed from raw distance. That is not GeoGuessr's own curve, which floors at zero past roughly 3,500 km.

How it behaves, which is the interesting part

It stopped exploring. Turns per episode fell from 6.7 to 1.1, output tokens from 1,062 to 66, and accuracy went up. It recognises the region from the first frame and commits to a plausible city in it, rather than reading signs across a dozen turns. That is not a random guess (a random guess on land averages roughly 8,000 km of error; this is 662 km) and it is not reward hacking: no identity leak in 882 audited prompts, no repeated tasks, and an action cost of 0.002 on the converged policy. It is what the reward rewarded, and the write-up is about why.

Training

Table

base	`Qwen/Qwen3.5-4B`
method	GRPO (TRL), `environment_factory` multi-turn tool calling
LoRA	r=16, α=32, dropout 0.05, on `q/k/v/o_proj`
steps	1000, one task per optimizer step, `NUM_GENERATIONS=8`
rollouts	8,000 episodes over 1,000 distinct tasks
turns	12 max
image

Note that it plateaued by step 200 (0.6393 against step 1000's 0.6445, inside the noise). If you reproduce this, stop at 250.

Using it

python
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.5-4B", device_map="auto")
model = PeftModel.from_pretrained(model, "HuggingEnvs/geoguesser-qwen3.5-4b-grpo")
processor = AutoProcessor.from_pretrained("Qwen/Qwen3.5-4B")

Or serve it with vLLM and point the eval harness at it:

bash
vllm serve Qwen/Qwen3.5-4B --enable-lora \
  --lora-modules run1=HuggingEnvs/geoguesser-qwen3.5-4b-grpo

Everything else

The write-up, which explains every decision and what it cost to learn it
The environment, playable in a browser
The task splits and the imagery
All four training runs in one dashboard
Code and exact commands

Imagery is Mapillary, CC BY-SA 4.0.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

HuggingEnvs

Model Tree

Base

Qwen/Qwen3.5-4B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality