Minbyul

AgentMercury-Qwen3.5-35B-A3B-SAO

README

License: apache-2.0

What it was trained to do

Every training episode is an autonomous investigation-and-write task against a synthetic enterprise MCP environment (email, chat, calendar, drive, HR, CRM, ticketing, …):

The model receives a persona and a single user request, and then gets no further user input — it must plan, search the environment, and act to completion on its own.
It acts multi-turn against a live tool surface (10–26 MCP tools per task, mean 16.2), up to 20 tool-use turns per episode.
Every task is state-changing (create / update / delete), so success is defined by the final state of the environment, not by the text of the answer.

Training corpus: 38,670 tasks over 3,865 distinct synthetic MCP environments — the hard split of the corpus, in which the required records are not visible from the opening prompt and have to be found first.

Reward: each rollout is graded by a rubric-based LLM judge over the trajectory, combined with assertions on the final environment state, and aggregated as the fraction of the required effects the episode actually achieved (dense partial credit, not pass/fail). A behaviour multiplier scales positive reward down for phrase repetition, post-answer runaway, and degenerate truncation, so that a clean trajectory and a rambling one are never rewarded the same.

So the direction of this training is narrow and deliberate: make the model finish real multi-step tool work correctly and without padding — not make it a better writer or a better solver of static questions.

Why SAO (what is different from the rest of the family)

The other AgentMercury checkpoints estimate advantages from a group of rollouts per prompt. This one follows SAO (arXiv:2607.07508) and takes one rollout per prompt, replacing the group baseline with a learned value model:

Table with columns: group-baseline RL, this run (SAO)
	group-baseline RL	this run (SAO)
rollouts per prompt	8	1
advantage baseline	mean reward of the group	learned critic (token-level GAE)
synchronisation	group barrier before learning	no barrier — a finished rollout enters the queue immediately
credit assignment	one scalar per trajectory	per-token, with observation tokens skipped

For long agentic episodes this matters: a 20-turn trajectory is mostly tool observations that the policy never generated, and a single trajectory-level scalar spreads credit over all of them. SAO's skip-observation GAE assigns value only to generated tokens, and dropping the group barrier removes the "wait for the slowest sibling rollout" cost, which is the dominant stall when episode lengths differ by 3×.

Concretely, as configured here:

Single-rollout sampling — group size 1, global batch 128 prompts.
Learned critic, initialised from the base model, with the whole token-mixing path (gated attention and the Gated DeltaNet linear mixer) frozen — only MLP/MoE, norms and the value head train.
Value pre-training: the first 20 steps update the critic only, so the policy never learns against an untrained baseline.
Faster value update: K=2 critic epochs per policy step.
Length-adaptive λ = 1 − 1/(α·l) with α=1.5, so GAE horizon scales with episode length; γ=1.0, λ_critic=1.
Decoupled trust-region surrogate with gradient masking outside the region (ε_low 0.2 / ε_high 0.28), which is what makes single-rollout updates stable off-policy.

Training configuration

Table

Base	Qwen3.5-35B-A3B (35B total / 3B activated, 256 experts, 40 hybrid layers)
Run length	200 rollout steps; these weights are step 159
Batch	128 prompts × 1 rollout
Episode budget	≤ 20 tool turns, ≤ 24,576 tokens (prompt + response)
Actor optimiser	Adam, lr 1e-6 constant, no warmup, wd 0, β = (0.9, 0.98), entropy coef 0
Critic optimiser	lr 5e-6, 10 warmup iters, 2 epochs/step, token-mixing frozen
Hardware	3 nodes × 8 GPUs — actor 8 (TP2, EP8) / critic 8 / rollout 8, fully asynchronous
Precision	bfloat16

Training signal at this checkpoint

Measured from the run's own rollout logs (step 159 = the state these weights were saved at):

Table with columns: base policy (steps 0–19), step 159
	base policy (steps 0–19)	step 159
reward (fraction of required effects)	0.348	0.649
all-effects pass rate	0.008	0.047
degenerate-generation rate	0.000	0.000
truncation rate (budget exhausted)	0.444	0.047
mean raw response length	~9.7k tokens	~12.9k tokens

The first 20 steps train the critic only, so their rollouts come from the unmodified base policy — that column is the base model measured in the same environment, by the same grader.

Reward rises monotonically across the run in 20-step windows (0.341 → 0.369 → 0.448 → 0.515 → 0.567 → 0.594 → 0.626 → 0.638) while the degeneration rate stays at zero on all but 2 of 168 rollout steps. The two length columns are the interesting part: the model gets longer on average (9.7k → 12.9k raw tokens) while the share of episodes that run out of budget collapses (44% → 5%). It is not learning to stop early; it is learning to spend its turns on work that terminates.

The strict metric — the share of episodes that achieve every required effect — is 4.7% at this step (4.2% averaged over steps 150–169), against 0.8% for the base policy. On this hard split partial completion is the norm; the dense reward is what makes the problem learnable at all.

These are training-environment numbers, not held-out benchmark scores. No external benchmark results are claimed for this checkpoint.

What these weights contain

Language decoder only. The RL run trained the language model, so the export carries model.language_model.* + lm_head (31,333 tensors, 69.3 GB bf16) and no model.visual.* tower and no MTP stack, while config.json is the base model's composite Qwen3_5MoeForConditionalGeneration config. Consequences:

Serve it text-only. A loader will report the vision tower as missing/newly-initialised; those weights are never executed for text prompts.
The MoE experts are stored unfused (...mlp.experts.N.gate_proj.weight), unlike the released base, which ships fused experts. SGLang handles both layouts and is the runtime this checkpoint was validated on.

Usage

SGLang (recommended — this is how the checkpoint was validated)

bash
python3 -m sglang.launch_server \
  --model-path Minbyul/AgentMercury-Qwen3.5-35B-A3B-SAO \
  --served-model-name agentmercury-sao-35b \
  --host 0.0.0.0 --port 30000 \
  --tp-size 1 --context-length 131072 --mem-fraction-static 0.85 \
  --moe-runner-backend triton --attention-backend triton \
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder \
  --trust-remote-code

Then call the OpenAI-compatible endpoint at http://localhost:30000/v1 (tool calls supported).

Serve with at least a 128K context window. Like its base, this is a thinking model, and it was trained toward longer trajectories (~13k tokens mean). A short window truncates generations inside the reasoning block and returns an empty response, which does not look like an error — it looks like a bad model. In our own sweep, serving at 32k made several benchmark scores numerically equal to "fraction of answers that happened to fit".

transformers

python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Minbyul/AgentMercury-Qwen3.5-35B-A3B-SAO",
    torch_dtype="bfloat16", device_map="auto", trust_remote_code=True,
)
tok = AutoTokenizer.from_pretrained(
    "Minbyul/AgentMercury-Qwen3.5-35B-A3B-SAO", trust_remote_code=True,
)

Limitations

Mid-run checkpoint. Step 159 of a 200-step run that was still improving when these weights were taken; it is not a converged or final model.
English, enterprise-tool domain. All RL tasks are English create/update/delete work over synthetic office-style services. Behaviour outside that distribution is the base model's.
Text-only export (see above) — no vision.
Longer outputs by construction. Reward favours thorough investigation, so responses are markedly longer than the base model's. Budget context and latency accordingly.
No held-out benchmark evaluation is reported for this checkpoint.

Citation

bibtex
@misc{agentmercury_sao_35b,
  title  = {AgentMercury-Qwen3.5-35B-A3B-SAO: single-rollout critic-based agentic RL for MCP tool use},
  author = {Jeong, Minbyul},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Minbyul/AgentMercury-Qwen3.5-35B-A3B-SAO}}
}

The RL objective follows SAO, arXiv:2607.07508.

License

Apache-2.0, inherited from the base model (see LICENSE).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Minbyul

Model Tree

Base

Qwen/Qwen3.5-35B-A3B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality