qwen3.5-9b-factorio-build-grpo API & Inference Endpoint

Results

Success rate (8 scenes per row, 16 rows × 4 samples, temperature 0.6, thinking off):

Table with columns: Model, Validation, Holdout
Model	Validation	Holdout
Qwen3.5-9B	2.1%	1.6%
SFT, 93 programs	18.0%	18.0%
SFT, 93 programs + 20 GRPO steps (this model)	93.8%	79.5%
SFT, 800 programs	67.0%	57.4%
SFT, 800 programs + 20 GRPO steps	85.9%	84.0%

GRPO adds 61.5 points on the holdout on top of this small warm start, more than it adds to the 800-program warm start (+27), but the validation-holdout gap is larger here (14 points against 2). Expect a few points of noise at this sample size.

Training

Start: the 93-program SFT model.
GRPO: LoRA r16 / alpha 32 on attention, Gated DeltaNet and MLP projections; AdamW lr 2e-5; 128 rollouts per step in groups of 16; temperature 0.7; thinking off; up to 12,288 completion tokens; groups with no reward spread filtered out; no KL term.
Reward: fraction of the row's 8 training scenes where the program verifies, plus 0.1 for a sandbox-valid program.
Compute: 2× A100 80GB (one vLLM, one trainer), about 2.5 minutes per step, stopped at step 20 (batch reward had plateaued near 0.9).

Recipe: recipes/grpo-factorio-build.

Limitations

One task (construct_smelting_line), one API, sandboxed; not a general coding model. The holdout includes scene families the training split does not sample.

Results

Success rate (8 scenes per row, 16 rows × 4 samples, temperature 0.6, thinking off):

Table with columns: Model, Validation, Holdout
Model	Validation	Holdout
Qwen3.5-9B	2.1%	1.6%
SFT, 93 programs	18.0%	18.0%
SFT, 93 programs + 20 GRPO steps (this model)	93.8%	79.5%
SFT, 800 programs	67.0%	57.4%
SFT, 800 programs + 20 GRPO steps	85.9%	84.0%

Training

Start: the 93-program SFT model.
GRPO: LoRA r16 / alpha 32 on attention, Gated DeltaNet and MLP projections; AdamW lr 2e-5; 128 rollouts per step in groups of 16; temperature 0.7; thinking off; up to 12,288 completion tokens; groups with no reward spread filtered out; no KL term.
Reward: fraction of the row's 8 training scenes where the program verifies, plus 0.1 for a sandbox-valid program.
Compute: 2× A100 80GB (one vLLM, one trainer), about 2.5 minutes per step, stopped at step 20 (batch reward had plateaued near 0.9).

Recipe: recipes/grpo-factorio-build.

Limitations

One task (construct_smelting_line), one API, sandboxed; not a general coding model. The holdout includes scene families the training split does not sample.

qwen3.5-9b-factorio-build-grpo

README

Results

Training

Limitations

Explore FriendliAI today

README

Results

Training

Limitations