Minbyul

Qwen3.5-35B-A3B-Correct-GRPO

README

License: apache-2.0

Model description

Architecture: Qwen3.5-35B-A3B — a mixture-of-experts (MoE) decoder-only transformer with ~35B total parameters and ~3B active per token. Architecture, tokenizer, and chat/tool-calling template are unchanged from the base model.
Lineage: Qwen3.5-35B-A3B → SFT on the correctness-filtered subset of an in-house web-search agent trajectory corpus (deterministic answer matching; Correct arm, 131,072-token training context) → turn-level GRPO (this model).
Behavior domain: multi-turn web-browsing agentic tool use (search / page-open / in-page find style tool surface) with explicit reasoning.

Training procedure (RL stage)

Turn-level GRPO on stop-decision pivot states: mid-trajectory states where the agent must decide between stopping to answer and continuing to search. Rollouts from each pivot state are rewarded by a deterministic stop-verifier (rule-based; no learned reward model).

Table with columns: Component, Value
Component	Value
Algorithm	GRPO (group-relative policy optimization), turn-level
Group size	K = 8 rollouts per pivot state
Reward	deterministic stop-verifier (rule-based; no learned reward model)
KL regularization	to the frozen SFT (Correct) reference policy, coefficient 0.01, low-variance KL estimator
Global batch size	128 sequences (rollout batches of 16 states x K = 8)
Released checkpoint	~14 effective RL steps, selected by a downstream step-sweep

Checkpoint selection and the base-dependent window

As with the Asis-based run, the RL run was swept checkpoint-by-checkpoint on downstream browsing-agent evaluation, and the released step is the accuracy-neutral over-search-reduction point for this base. The step-sweep found that on the correctness-filtered initialization, accuracy regression sets in several times earlier and more sharply than on the unfiltered Asis initialization — hence the much earlier selected checkpoint. This asymmetry is a central observation of the study: the same stop-decision RL recipe has a base-dependent safe window, and checkpoints (or step budgets) must not be transplanted across initializations.

Intended use

Research artifact. Intended for studying how the SFT initialization shapes the safe window of stop-decision RL, and for arm-invariance / generality analyses alongside its Asis-based counterpart. It expects a browsing-agent tool surface compatible with its training format (function-calling tools for web search, page opening, and in-page lookup).

Limitations

Research artifact, not a production assistant; behavior is tuned for a specific browsing-agent tool surface and prompt format.
Narrow safe window. This base tolerates very little stop-decision RL before regressing; the released checkpoint is close to that edge. Any continued training from this checkpoint should re-run the step-sweep rather than reuse the Asis-base schedule.
The stop-verifier is a rule-based proxy for good stopping behavior; efficiency characteristics should be re-validated under any new tool surface or reward definition.
Only a small number of RL steps were applied; capabilities outside the stop-decision behavior are essentially those of the SFT parent.
No additional safety alignment was applied beyond what the base model provides.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Minbyul

Model Tree

Base

Qwen/Qwen3.5-35B-A3B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality