divagr1925

qwen3.5-9b-factorio-build-er-r1

README

License: apache-2.0

Evolve & Reinforce

Evolve. LLM-guided program search produces builder programs that pass on training scenes.
Distil in the model's own voice. Base Qwen3.5-9B is shown each evolved program and asked to explain the approach step by step and write the program itself. A reply is kept only if its program scores at least as well as the original: 279 of 800. Training pairs the prompt without the program with the model's own reply, so the targets stay close to what the model already writes.
Light SFT (LoRA rank 32, 3 epochs, lr 1e-4) on those replies, plus 20% replay of the base model's own answers to general prompts: qwen3.5-9b-factorio-build-er-r1-sft.
Reinforce. 30 GRPO steps with prime-rl (LoRA rank 16, group size 16, 128 rollouts per step, temperature 0.7). Train success went from 23% to 86%.

Prompt version v2 of factorio-build (reason step by step, then the program), thinking off.

Results

Factorio success: 8 scenes per row, 16 rows × 4 samples, temperature 0.6, thinking off. PlanBench Blocksworld (500 instances each, thinking off, temperature 0) measures general planning.

Table with columns: Model, Factorio val, Factorio holdout, PlanBench plan generation, PlanBench plan execution, Reply tokens (plan gen.)
Model	Factorio val	Factorio holdout	PlanBench plan generation	PlanBench plan execution	Reply tokens (plan gen.)
Qwen3.5-9B (base)	2.3%	1.6%	53.2%	39.8%	1,975
SFT on 800 bare programs + 20 GRPO steps (earlier recipe)	85.9%	84.0%	2.8%	3.0%	64
Own-voice SFT (round 1 warm start)	22.3%	–	49% (first 100)	–	1,859
Own-voice SFT + 30 GRPO steps (this model)	85.6%	71.9%

The earlier recipe fine-tuned on another model's bare programs, and the model stopped reasoning before answering (64-token replies) and lost general planning. This recipe keeps the base model's reasoning and most of its planning ability, at a lower holdout score. Plan execution still drops by 12 points, which the next round targets.

Use

python

messages = [{"role": "system", "content": system_prompt}, {"role": "user", "content": user_prompt}]# chat_template_kwargs={"enable_thinking": False}, temperature=0.6

The prompts come from the factorio-build environment (prompt_version="v2").

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

divagr1925

Model Tree

Base

divagr1925/qwen3.5-9b-factorio-build-er-r1-sft

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality