Lego-X

qwen3_5_35b_a3b_ohsdk_200k_rl

README

License: apache-2.0

Why harness-native training

The scaffold is part of the environment, not the policy. The same weights score very differently depending on which harness runs them, so training under a rewritten control flow optimizes for a deployment you never ship:

Table with columns: Model, OpenHands SDK, Claude Code, OpenCode
Model	OpenHands SDK	Claude Code	OpenCode
Qwen3.5-35B-A3B (starting point)	64.0	62.4	57.2
Qwen3.6-35B-A3B (next-gen base)	67.4	63.4	60.6
KAT-Coder-V2.5-Dev (post-trained Qwen3.6)	67.0	66.8	64.8
Lego-RL-Qwen3.5-35B-A3B	70.4	68.2	66.6

SWE-bench Verified (%), one shared protocol: temperature 0.7, 200 turns, 200K context.

Each Lego-RL column is a separate run trained in that harness from the same starting checkpoint, the same 2,699 tasks and the same 3 epochs. This repository is the OpenHands SDK run (70.4). The Claude Code and OpenCode runs are reported in the paper; their checkpoints are being released into the Lego-RL collection. Across the three harnesses RL adds +6.4 / +5.8 / +9.4 — more than the entire Qwen3.5 → Qwen3.6 base-model generation (+3.4 / +1.0 / +3.4).

Quick start

1. Serve with vLLM

bash
vllm serve Lego-X/qwen3_5_35b_a3b_ohsdk_200k_rl \
    --served-model-name vllm_model \
    --tensor-parallel-size 4 \
    --enable-expert-parallel \
    --max-model-len 262144 \
    --gpu-memory-utilization 0.9 \
    --enable-chunked-prefill --enable-prefix-caching \
    --dtype bfloat16 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder \
    --host 0.0.0.0 --port 8000

[!IMPORTANT] --tool-call-parser qwen3_coder is not optional. The model was rolled out and trained with this parser; serving it behind hermes (or any other parser) silently degrades tool-call formatting — this exact misconfiguration cost us a full cluster-day during development.

2. Drive it with OpenHands SDK

The model is a coding agent policy: it expects a harness that gives it a repository, a shell and file-edit tools, and it expects room to run. Point OpenHands SDK at the vLLM endpoint as an OpenAI-compatible provider:

python
from openhands.sdk import LLM, Agent, Conversation   # openhands-sdk >= 1.14
from pydantic import SecretStr

llm = LLM(
    model="openai/vllm_model",
    base_url="http://localhost:8000/v1",
    api_key=SecretStr("dummy"),
    temperature=0.7,
    max_input_tokens=200_000,
    max_output_tokens=32_768,
)
agent = Agent(llm=llm)
conversation = Conversation(agent=agent, workspace="/path/to/repo")
conversation.send_message("Fix the failing test in tests/test_units.py")
conversation.run()

Consult your openhands-sdk version's own docs for the exact constructor signature — the training runs used the openhands-sdk runtime image c-oh-sdk-1.33.0, and evaluation used c-oh-sdk-1.14.0.

Run it with 200 turns and 200K context. The RL policy learned to spend turns: mean turns per trajectory grew from 46.6 to 83.1. Capping turns at the usual 50–100 truncates the second half of its trajectories systematically and costs most of the gain.

3. Reproduce the evaluation

bash
git clone https://github.com/LegoX/Lego-RL.git && cd Lego-RL
bash scripts/setup_env.sh
cp scripts/eval/_template.env scripts/eval/configs/my_eval.env   # set MODEL_PATH, DATASET_PATH, kubeconfig
bash scripts/eval/eval.sh scripts/eval/configs/my_eval.env

Sandboxed execution and verifier rewards come from Harbor; see the evaluation docs.

Training

Table

Starting checkpoint	Qwen/Qwen3.5-35B-A3B (sparse MoE, 256 experts, 8 active)
Harness	OpenHands SDK, unmodified — a thin adapter, not a fork
Tasks	Lego-X/Lego-RL-2699 — 2,699 real repository issues, converted from GAIR/OpenSWE
Reward	each task's own test suite, run in a fresh sandbox: `{0, 1}`. No reward model, no patch similarity, no LLM judge
Algorithm	GSPO (sequence-level surrogate), group-relative advantage over rollouts per task

Three properties made this run trainable rather than merely runnable:

Faithful capture. An in-process proxy at the serving boundary records token ids, masks, log-probs and expert routes at generation time — nothing is re-tokenized from a stored transcript, which matters because the harness rewrites its own history (compaction, re-serialized tool arguments, injected reminders). Replaying rollout-time MoE routing (R3) lifted rollout↔training correlation from 0.9946 to 0.9993; across all three production runs the median per-step correlation stayed ≥ 0.998.
Reliable execution. Reward-hacking paths are closed inside the environment, per phase: no network and no fix-bearing git history during the agent phase, both restored for grading; test files are withheld and any edit to them rolled back. Trajectories that died of infrastructure faults (2.4% here) are masked out of the loss rather than scored as failures.
Difficulty-filtered task pool. 36,884 OpenSWE candidates → 2,699 (12.4% of the executable pool), keeping only tasks solved 1–3 times out of 4 by a reference rollout model. An unfiltered pool never improved at all: 72.7% of its tasks were never solved and 13.4% always were, so under six tasks in a hundred produced any in-group reward variance.

The training set is disjoint from SWE-bench Verified at both the repository and the instance level.

What RL actually changed

420 trajectories from the first step vs. 420 from the last, in the OpenHands SDK run:

Table with columns: Behavior, Before, After
Behavior	Before	After
Re-reads a file after editing it	73.6%	98.1%
Files inspected before the first edit	3.45	6.92
Runs the test suite on its own initiative	85.0%	93.6%
Mean turns per trajectory	46.6	83.1
Malformed tool calls	1.07%	0.15%
pass@8

The honest reading: the model became more reliable on tasks it could already sometimes solve, more than it became capable of new ones. pass⁸ moved 11.1 points while pass@8 moved 4.7. Mid-trajectory error recovery barely moved (63.9% → 66.8%) — a terminal binary reward gives no independent signal for recovering from a failed command.

Intended use and limitations

Use it as an agent policy, not as a chat model: it was optimized inside a harness that hands it a repository, a shell, and file-editing tools, and its behavior (long inspect-edit-verify loops, heavy test invocation) only pays off in that setting.

Harness. Trained in OpenHands SDK. It also runs under Claude Code / OpenCode / any OpenAI- or Anthropic-compatible harness, but the paper trains a separate policy per harness, and each does best in the one it was trained in.
Budget. 200K context and 200 turns; response lengths roughly doubled during training (43.5K → 90.9K tokens per trajectory). Short budgets truncate it.
Domain. Python-heavy repository issue-resolution, in the SWE-bench/OpenSWE distribution. No claim outside it.
Inherited base behavior. Safety, multilingual and general-knowledge behavior come from Qwen3.5-35B-A3B and were not targeted by this RL. The architecture retains the base model's vision tower; training and evaluation were text-only.
Sandbox it. The policy writes files and executes shell commands on purpose. Run it in a container.

Table

Lego-RL	harness-native RL for coding agents (this model)
SWE-Lego	the SFT recipe
SWE-Review	inference-time generate-review-revise
Terminal-Lego	trajectory-quality filtering

Acknowledgement

Built on verl (trainer + rollout) and Harbor (sandboxed execution + verifier reward), with OpenHands as the harness.

Citation

bibtex
@misc{du2026legorlharnessnativereinforcementlearning,
  title={LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents},
  author={Yiming Du and Yuxin Jiang and Tao Yuan and Jianbo Dai and Shaowei Wang and Jierun Chen and Chaofan Tao and Xianzhi Yu and Lifeng Shang and Kam-Fai Wong and Xiaohui Li and Haoli Bai},
  year={2026},
  eprint={2608.17393},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2608.17393},
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Lego-X

Model Tree

Base

Qwen/Qwen3.5-35B-A3B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Why harness-native training

Table with columns: Model, OpenHands SDK, Claude Code, OpenCode
Model	OpenHands SDK	Claude Code	OpenCode
Qwen3.5-35B-A3B (starting point)	64.0	62.4	57.2
Qwen3.6-35B-A3B (next-gen base)	67.4	63.4	60.6
KAT-Coder-V2.5-Dev (post-trained Qwen3.6)	67.0	66.8	64.8
Lego-RL-Qwen3.5-35B-A3B	70.4	68.2	66.6

SWE-bench Verified (%), one shared protocol: temperature 0.7, 200 turns, 200K context.

Quick start

1. Serve with vLLM

bash
vllm serve Lego-X/qwen3_5_35b_a3b_ohsdk_200k_rl \
    --served-model-name vllm_model \
    --tensor-parallel-size 4 \
    --enable-expert-parallel \
    --max-model-len 262144 \
    --gpu-memory-utilization 0.9 \
    --enable-chunked-prefill --enable-prefix-caching \
    --dtype bfloat16 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder \
    --host 0.0.0.0 --port 8000

[!IMPORTANT] --tool-call-parser qwen3_coder is not optional. The model was rolled out and trained with this parser; serving it behind hermes (or any other parser) silently degrades tool-call formatting — this exact misconfiguration cost us a full cluster-day during development.

2. Drive it with OpenHands SDK

python
from openhands.sdk import LLM, Agent, Conversation   # openhands-sdk >= 1.14
from pydantic import SecretStr

llm = LLM(
    model="openai/vllm_model",
    base_url="http://localhost:8000/v1",
    api_key=SecretStr("dummy"),
    temperature=0.7,
    max_input_tokens=200_000,
    max_output_tokens=32_768,
)
agent = Agent(llm=llm)
conversation = Conversation(agent=agent, workspace="/path/to/repo")
conversation.send_message("Fix the failing test in tests/test_units.py")
conversation.run()

3. Reproduce the evaluation

bash
git clone https://github.com/LegoX/Lego-RL.git && cd Lego-RL
bash scripts/setup_env.sh
cp scripts/eval/_template.env scripts/eval/configs/my_eval.env   # set MODEL_PATH, DATASET_PATH, kubeconfig
bash scripts/eval/eval.sh scripts/eval/configs/my_eval.env

Sandboxed execution and verifier rewards come from Harbor; see the evaluation docs.

Training

Table

Starting checkpoint	Qwen/Qwen3.5-35B-A3B (sparse MoE, 256 experts, 8 active)
Harness	OpenHands SDK, unmodified — a thin adapter, not a fork
Tasks	Lego-X/Lego-RL-2699 — 2,699 real repository issues, converted from GAIR/OpenSWE
Reward	each task's own test suite, run in a fresh sandbox: `{0, 1}`. No reward model, no patch similarity, no LLM judge
Algorithm	GSPO (sequence-level surrogate), group-relative advantage over rollouts per task

Three properties made this run trainable rather than merely runnable:

Faithful capture. An in-process proxy at the serving boundary records token ids, masks, log-probs and expert routes at generation time — nothing is re-tokenized from a stored transcript, which matters because the harness rewrites its own history (compaction, re-serialized tool arguments, injected reminders). Replaying rollout-time MoE routing (R3) lifted rollout↔training correlation from 0.9946 to 0.9993; across all three production runs the median per-step correlation stayed ≥ 0.998.
Reliable execution. Reward-hacking paths are closed inside the environment, per phase: no network and no fix-bearing git history during the agent phase, both restored for grading; test files are withheld and any edit to them rolled back. Trajectories that died of infrastructure faults (2.4% here) are masked out of the loss rather than scored as failures.
Difficulty-filtered task pool. 36,884 OpenSWE candidates → 2,699 (12.4% of the executable pool), keeping only tasks solved 1–3 times out of 4 by a reference rollout model. An unfiltered pool never improved at all: 72.7% of its tasks were never solved and 13.4% always were, so under six tasks in a hundred produced any in-group reward variance.

The training set is disjoint from SWE-bench Verified at both the repository and the instance level.

What RL actually changed

420 trajectories from the first step vs. 420 from the last, in the OpenHands SDK run:

Table with columns: Behavior, Before, After
Behavior	Before	After
Re-reads a file after editing it	73.6%	98.1%
Files inspected before the first edit	3.45	6.92
Runs the test suite on its own initiative	85.0%	93.6%
Mean turns per trajectory	46.6	83.1
Malformed tool calls	1.07%	0.15%
pass@8

Intended use and limitations

Harness. Trained in OpenHands SDK. It also runs under Claude Code / OpenCode / any OpenAI- or Anthropic-compatible harness, but the paper trains a separate policy per harness, and each does best in the one it was trained in.
Budget. 200K context and 200 turns; response lengths roughly doubled during training (43.5K → 90.9K tokens per trajectory). Short budgets truncate it.
Domain. Python-heavy repository issue-resolution, in the SWE-bench/OpenSWE distribution. No claim outside it.
Inherited base behavior. Safety, multilingual and general-knowledge behavior come from Qwen3.5-35B-A3B and were not targeted by this RL. The architecture retains the base model's vision tower; training and evaluation were text-only.
Sandbox it. The policy writes files and executes shell commands on purpose. Run it in a container.

Table

Lego-RL	harness-native RL for coding agents (this model)
SWE-Lego	the SFT recipe
SWE-Review	inference-time generate-review-revise
Terminal-Lego	trajectory-quality filtering

Acknowledgement

Built on verl (trainer + rollout) and Harbor (sandboxed execution + verifier reward), with OpenHands as the harness.

Citation

bibtex
@misc{du2026legorlharnessnativereinforcementlearning,
  title={LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents},
  author={Yiming Du and Yuxin Jiang and Tao Yuan and Jianbo Dai and Shaowei Wang and Jierun Chen and Chaofan Tao and Xianzhi Yu and Lifeng Shang and Kam-Fai Wong and Xiaohui Li and Haoli Bai},
  year={2026},
  eprint={2608.17393},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2608.17393},
}

qwen3_5_35b_a3b_ohsdk_200k_rl

README

Why harness-native training

Quick start

1. Serve with vLLM

2. Drive it with OpenHands SDK

3. Reproduce the evaluation

Training

What RL actually changed

Intended use and limitations

Related work in the LegoX series

Acknowledgement

Citation

Explore FriendliAI today

README

Why harness-native training

Quick start

1. Serve with vLLM

2. Drive it with OpenHands SDK

3. Reproduce the evaluation

Training

What RL actually changed

Intended use and limitations

Related work in the LegoX series

Acknowledgement

Citation