C12H18

gats-coldstart-opd-grpo-qwen2.5-7b-webshop

README

License: apache-2.0

Chain: pure-OPD cold-start -> pure GRPO

Starting point: the pure-OPD cold-start checkpoint (see C12H18/gats-coldstart-opd-qwen2.5-7b-webshop, N = 29 on-policy distillation steps). Then pure GRPO:

GRPO: 150 steps, lr 1e-6, group n=8, 16 tasks/step, gamma 0.95, KL loss (low-variance, coef 0.01), invalid-action penalty 0.1, entropy 0.001, grad clip 1.0
Run id: opdgrpo-ws7b-t1p5b-s0 (seed 0)
Final evaluation: 72.48 ± 5.64 (K=3, rollout seeds 0/1/2 × env seeds 0/1/2)

Intended use

Research on cold-start strategies for agent RL (imitation-first vs RL-from-scratch) and reproduction of the GATS paper baselines. Not a general-purpose assistant.

Provenance

Trained 2026-09-28 on the internal verl-based stack; checkpoints uploaded unmodified from the training runs (bfloat16 unless noted float32 above).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

C12H18

Model Tree

Base

Qwen/Qwen2.5-7B-Instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities