Final evaluation (step 150)
Standalone vLLM harness, 128 ALFWorld valid_seen tasks, 3 seeds, temperature 0.4,
max 50 env steps:
Table with columns: value | value |
|---|
| success rate | 84.38% ± 1.69 |
base model (Qwen2.5-1.5B-Instruct) | 2.86% ± 0.37 |
| mean episode length | 16.42 (base: 49.15) |
| format rate | 99.49% (base: 98.12%) |
Training curve
In-training validation success rate (val/success_rate), measured every 5 optimizer
steps on a 32-task validation batch. Note this is a smaller and noisier probe than the
final 128-task evaluation above.
Table with columns: optimizer step, success rate (%)| optimizer step | success rate (%) |
|---|
| 5 | 6.2 |
| 40 | 29.7 |
| 50 | 40.6 |
| 55 | 35.2 |
| 60 | 49.2 |
| 65 | 57.0 |
| 70 | 51.6 |
| 75 | 54.7 |
| 80 | 51.6 |
|
Hyperparameters
Table with columns: group, parameter, value| group | parameter | value |
|---|
| RL | adv_estimator | gigpo |
| RL | gigpo.mode | mean_std_norm |
| RL | gigpo.step_advantage_w | 1.0 |
| RL | algorithm.gamma | 0.95 |
| RL |
Files
Table with columns: path, contents| path | contents |
|---|
model.safetensors, config.json, tokenizer files | bf16 weights, merged from the FSDP shards with scripts/model_merger.py. lm_head is omitted because tie_word_embeddings=true (same layout as the base Qwen release). |
training_state/model_world_size_2_rank_{0,1}.pt | FSDP-sharded weights |
training_state/optim_world_size_2_rank_{0,1}.pt | Adam optimizer state |
training_state/extra_state_world_size_2_rank_{0,1}.pt |
training_state/ lets you resume RL training from this exact optimizer step. The shards
are written for world_size=2; resuming on a different number of GPUs requires
resharding.
Available checkpoints
step 130 · step 140 · step 150
Only steps 130/140/150 survive: the run used trainer.max_actor_ckpt_to_keep=3, so
earlier checkpoints were pruned during training.
Comparison caveat
A StraTA reproduction on the same task and backbone is published alongside these models
(chanyoungkim/strata-qwen2.5-1.5b-alfworld-step*). The two are not directly comparable —
they use different agent scaffolds:
Table with columns: GiGPO (this model), StraTA | GiGPO (this model) | StraTA |
|---|
| eval tasks | 128 (valid_seen subset) | 140 (valid_seen, all) |
| temperature | 0.4 | 0.7 |
max_prompt_length | 2048 | 7168 |
max_response_length | 512 | 1024 |
| prompt format |
To compare the methods fairly, re-evaluate both under a single scaffold.
Caveats
- Do not use
is_correct / pass@1 from verl logs; they are hardcoded to 1.0.
Use val/average/metrics/success_rate (or val/success_rate in verl-agent).
- Evaluated on
valid_seen only; valid_unseen was not run.
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "chanyoungkim/gigpo-qwen2.5-1.5b-alfworld-step140"
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")
tok = AutoTokenizer.from_pretrained(repo)