code-critic-model

Qwen3-8B-Critic-SFT

README

License: apache-2.0

Where it appears in the paper

Table with columns: Paper location, Row label
Paper location	Row label
Table 1, every agent block	`Qwen3-8B + SFT`
Table 2, Multi-SWE-bench	`+ SFT`
Table 3, corpus ablation	`8B, Qwen+CWM`
Table 4, prompt ablation	`+ SFT (High-Level, Ours)`
Figure 4, cost versus resolve rate	the 8B SFT critic marker for each agent

Original run name: qwen3-8b-full-sft-prm-r2egym-swebench-instructions-k5-cwm-plus-qwen-opus-distill-32k-lr5e6-multiturn. This is the name that appears in the repository's configs, logs, result directories, and the LiteLLM cost registry (there under the shubhamrgandhi/ prefix).

Training data

code-critic-model/critic-sft-cwm-qwen, 6,447 examples.

Tasks: 500 R2E-Gym instances from the matplotlib, moto, and sympy repositories. These are disjoint from SWE-bench Verified at the instance level.
Agents that produced the trajectories: CWM-32B (500 trajectories, 4,532 examples) and Qwen3-Next-80B-A3B-Instruct (483 trajectories, 1,915 examples).
Teacher: Claude Opus 4.6, queried every 5 agent steps with the high-level prompt. The high-level prompt asks for error detection and short guidance and forbids full code solutions. The paper calls this the "high-level" or "concise" prompt.

Training setup

Full-parameter SFT with LLaMA-Factory. The config is finetuning/qwen3_8b_critic_full_sft_l40s_train_multiturn_resumable.yaml in the repository.

Table with columns: Setting, Value
Setting	Value
Base model	Qwen/Qwen3-8B
Chat template	`qwen3_nothink` (thinking disabled at training and inference)
Sequence length	32,768 tokens
Loss	final critique turn only (`mask_history: true`)
Hardware	8 x L40S, per-device batch 1, effective batch 8
Optimizer	AdamW, lr 5e-6, cosine schedule, warmup ratio 0.1
Epochs	3
Precision

Results

Resolve rate on SWE-bench Verified (500 instances), no critic versus this critic. Numbers are from Table 1 of the paper and take the better of k=5 and k=10 for each configuration.

Table with columns: Coding agent, No critic, + Qwen3-8B-Critic-SFT
Coding agent	No critic	+ Qwen3-8B-Critic-SFT
Qwen3-32B	8.8	13.8
Qwen3-Next-80B-A3B	20.0	25.2
GPT-OSS-20B	3.0	13.0
GLM-4.7-Flash-30B-A3B	21.6	37.6
GPT-OSS-120B (medium reasoning)	20.4	31.4
o3-mini	19.0

Resolve rate on 300 Multi-SWE-bench instances (Table 2), with k=5. The critic saw only Python trajectories during training.

Table with columns: Coding agent, No critic, + Qwen3-8B-Critic-SFT
Coding agent	No critic	+ Qwen3-8B-Critic-SFT
Qwen3-Next-80B-A3B	9.7	11.7
Qwen3-32B	1.3	3.3

How to use

The critic was served with vLLM in bf16 and called through the repository's fork of mini-swe-agent, which inserts a critique into the agent's context every k steps.

bash
vllm serve code-critic-model/Qwen3-8B-Critic-SFT \
    --served-model-name Qwen3-8B-Critic-SFT \
    --dtype bfloat16 --max-model-len 65536 --port 8071

Then, from the repository root, run an agent with the step-aware critic prompt at k=5:

bash
bash scripts/run_critic_max150.sh prm_issue_res_instructions_step_aware 5 0 qwen3-80b \
    --prm Qwen3-8B-Critic-SFT --prm-node <vllm-host>:8071 --slice :500 \
    --prefix-dir <path to the matching no-critic run>

The launcher passes the --prm name to LiteLLM, which needs a matching entry in mini-swe-agent/configs/litellm_model_registry.json to price the calls. Copy the block for the original run name to a new key Qwen3-8B-Critic-SFT, or serve under the original run name instead. Without a registry entry the critic call fails and the agent runs without critiques. The full inference procedure, including the agent-side configs and the no-AWS path, is in QUICKSTART.md and HANDOVER.md.

To call the critic directly, reuse a training record as the prompt. The system message and the trajectory encoding are exactly what the model saw during training.

python
from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "code-critic-model/Qwen3-8B-Critic-SFT"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto")

record = load_dataset("code-critic-model/critic-sft-cwm-qwen", split="train")[0]
messages = record["messages"][:-1]   # drop the teacher critique, keep system + trajectory
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,
                                 return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=1024, do_sample=False)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Limitations

The critic was trained on Python repositories only and on trajectories in the mini-swe-agent format (one bash command per step). It has been evaluated as a critic for other agents, not as a stand-alone coder, and its critiques are only as reliable as the teacher's on the training distribution.

Citation

bibtex
@misc{gandhi2026steerdontsolvetraining,
  title={Steer, Don't Solve: Training Small Critic Models for Large Code Agents},
  author={Shubham Gandhi and Yiqing Xie and Atharva Naik and Ruichen Zhu and Carolyn Rose},
  year={2026},
  eprint={2606.21811},
  archivePrefix={arXiv},
  primaryClass={cs.SE},
  url={https://arxiv.org/abs/2606.21811}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

code-critic-model

Model Tree

Base

Qwen/Qwen3-8B

Fine-tuned

this model

Input Modalities

Text

Output Modalities