Training setup
- Base model: Qwen/Qwen2.5-1.5B-Instruct; weights here are full fp32 HF exports (actor only — GRPO has no critic)
- Algorithm: GRPO,
rollout.n=8, train_batch 8, prompt/response 4096/512, fused kernels (triton)
- Federation: 100 clients, 2 sampled per round, 70 rounds, 3 local epochs/round, FedAvg over actor weights,
base_seed 42 (fully deterministic schedule)
- Task heterogeneity: hardness partition, success_std = 1 — each client's WebShop goal pool is stratified by a trained-reference success signal (≥100 goals/client)
- Environment: vendored WebShop engine, BM25 search (
search_return_n=50), 16-env pools per client, unperturbed shared val service
- Hardware / wall-clock: 4× H100-80GB single node, ~16.5 min/round, ~19 h total
- Config: run copy of
config/paper_accelerated/task_heterogeneity/grpo/webshop/fed_webshop_grpo_total-100_cl-per-rd-2_rd-70_ep-per-cl-3_min-goals-per-cl-100_p-hardness_success_std-1.yaml (science knobs untouched)
The full run record — per-round validation dumps, training logs, curves, and the official
federated_summary.json (71 global val points + 140 per-client circles) — is published
separately as a dataset: canyuchen/fedagent-webshop-grpo-hardness-std1.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "canyuchen/fedagent-webshop-grpo-hardness-std1-qwen2.5-1.5b"
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained(repo)
The agent expects WebShop-style observation/action prompting (as produced by the FedAgent
rollout loop); it is not a general chat model, though the chat template is included.
Note on chat_template: verl's model_merger drops the tokenizer's chat_template when
exporting; it has been re-injected from the base Qwen tokenizer here, so
tok.apply_chat_template(...) works out of the box. If you re-export checkpoints from the raw
run artifacts yourself, remember to do the same.
Provenance
- Trained 2026-07-23 → 2026-07-24 (fresh run, no prior checkpoints), FedAgent verl-0.8 overlay
- Companion PPO twin (same partition/protocol) reached only 14.1 % best success by round 34 —
GRPO dominates at every matched round
- fp32 shards, ~5.8 GB per checkpoint;
model.safetensors.index.json included