Minbyul

Qwen3.5-35B-A3B-Asis-GRPO

README

License: apache-2.0

Model description

Architecture: Qwen3.5-35B-A3B — a mixture-of-experts (MoE) decoder-only transformer with ~35B total parameters and ~3B active per token. Architecture, tokenizer, and chat/tool-calling template are unchanged from the base model.
Lineage: Qwen3.5-35B-A3B → SFT on an in-house web-search agent trajectory corpus, used unmodified (Asis arm, 131,072-token training context) → turn-level GRPO (this model).
Behavior domain: multi-turn web-browsing agentic tool use (search / page-open / in-page find style tool surface) with explicit reasoning.

Training procedure (RL stage)

Turn-level GRPO on stop-decision pivot states: mid-trajectory states where the agent has just produced (or is about to produce) enough evidence to answer, and must decide whether to stop or keep searching. Rollouts are generated from each pivot state and rewarded by a deterministic stop-verifier — a rule-based checker of whether the continuation stops appropriately and remains grounded — rather than by a learned reward model.

Table with columns: Component, Value
Component	Value
Algorithm	GRPO (group-relative policy optimization), turn-level
Group size	K = 8 rollouts per pivot state
Reward	deterministic stop-verifier (rule-based; no learned reward model)
KL regularization	to the frozen SFT (Asis) reference policy, coefficient 0.01, low-variance KL estimator
Global batch size	128 sequences (rollout batches of 16 states x K = 8)
Released checkpoint	~32 effective RL steps, selected by a downstream step-sweep

Checkpoint selection

The RL run was intentionally trained past its useful region and swept checkpoint-by-checkpoint on downstream browsing-agent evaluation. The released step (~32) is the accuracy-neutral over-search-reduction point: the latest checkpoint at which end-task accuracy is statistically indistinguishable from the SFT baseline while post-answer over-search (continued searching after the answer has been stated) is significantly reduced. Later checkpoints trade accuracy or healthy verification behavior for further efficiency and were not released.

Intended use

Research artifact. Intended for studying stop-decision RL, over-search reduction, and the accuracy-efficiency frontier of tool-using web agents; also usable as a more search-efficient drop-in for the Asis SFT arm on the same tool surface (function-calling tools for web search, page opening, and in-page lookup).

Limitations

Research artifact, not a production assistant; behavior is tuned for a specific browsing-agent tool surface and prompt format.
The safe RL window is base-dependent. The ~32-step selection is specific to this SFT initialization; on other initializations the beneficial window opens and closes at different points (see Qwen3.5-35B-A3B-Correct-GRPO, whose sweet spot is much earlier). Do not assume the step choice, or the checkpoint itself, transfers across bases.
The stop-verifier is a rule-based proxy for good stopping behavior; the model may have partially adapted to the verifier's notion of evidence access, so efficiency gains should be re-validated under any new tool surface or reward definition.
Only a small number of RL steps were applied; capabilities outside the stop-decision behavior are essentially those of the SFT parent.
No additional safety alignment was applied beyond what the base model provides.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Minbyul

Model Tree

Base

Qwen/Qwen3.5-35B-A3B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality