Training
The training task pool contained 2,420 ALFWorld task IDs. Training ran for 200 steps with 16 task prompts per step and 8 sampled environment rollouts per prompt (25,600 training episodes in total). This was full-parameter GRPO, with learning rate 1e-6, KL-loss coefficient 0.001 and seed 42. The policy used multi-turn ReAct-style text actions and native environment feedback, without a learned reward model or expert action demonstrations. No cross-domain synthetic SFT was applied to this checkpoint.
Native Qwen3 thinking mode was disabled during training. The action protocol may still contain ordinary Thought: text. Training allowed at most 30 environment turns, 512 newly generated tokens per turn, and 8,192 generated tokens per episode. See training_config.json for the sanitized configuration and training_metrics.json for the sampled training-reward curve.
Training reward reached a high plateau, but training stopped at a predetermined budget of 200 steps. Periodic held-out validation was disabled. This is not evidence that further GRPO training cannot improve performance.
Historical ALFWorld evaluation
On the project's fixed AgentGym 200-task ALFWorld test set, the original Qwen3-4B scored 30.0625 and this checkpoint scored 89.1875. The metric is mean native reward across 8 sampled rollouts per task, averaged across tasks and multiplied by 100; it is not pass@8.
This evaluation disabled native thinking, allowed 20 environment turns, and used the project's VERL ReAct driver with pinned AgentGym environments. The test tasks come from ALFWorld valid_train; they are not the ALFWorld unseen split. These historical results should not be treated as an independent reproduction through an official evaluation entrypoint, or compared directly with scores using different splits, prompts, budgets or thinking settings. See eval_results.json.
Loading
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "PassionPrc/Qwen3-4B-ALFWorld-GRPO-step200"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto") # Supply the actual ALFWorld task, environment observation and action protocol.messages = [{"role": "user", "content": observation_with_task_and_action_instructions}]input_ids = tokenizer.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, enable_thinking=False, return_tensors="pt").to(model.device)output = model.generate(input_ids, max_new_tokens=512)reply = tokenizer.decode(output[0, input_ids.shape[1]:], skip_special_tokens=True)
The model emits action text. A separate ALFWorld environment and agent loop must parse the action, execute it, append the real observation and request the next action. Loading the weights alone does not provide an environment. Native thinking can be requested through the tokenizer, but the reported result above does not validate that setting for this checkpoint.
Files and provenance
The two BF16 safetensors shards were exported from the final four-rank FSDP actor using VERL's model merger. Their tensor headers and shard index were checked before release. release_manifest.json contains file sizes and SHA256 hashes. The tokenizer and model configuration accompany the weights. Optimizer state, raw trajectories, credentials and internal filesystem paths are not included.
The base model is distributed under Apache-2.0; this release includes that license. Relevant projects: Qwen3, VERL, AgentGym-RL, and ALFWorld.