C12H18

gats-coldstart-opd-grpo-qwen2.5-7b-sciworld

README

License: apache-2.0

Chain: pure-OPD cold-start -> pure GRPO

Starting point: the pure-OPD cold-start checkpoint (see C12H18/gats-coldstart-opd-qwen2.5-7b-sciworld, N = 29 on-policy distillation steps). Then pure GRPO:

GRPO: 150 steps, lr 1e-6, group n=8, 16 tasks/step, gamma 0.95, KL loss (low-variance, coef 0.01), invalid-action penalty 0.1, entropy 0.001, grad clip 1.0
Run id: opdgrpo-sci7b-t1p5b-s0 (seed 0)
Final evaluation: seen 37.76 ± 2.51 / unseen 33.07 ± 3.93 (K=3, rollout seeds 0/1/2)

Intended use

Research on cold-start strategies for agent RL (imitation-first vs RL-from-scratch) and reproduction of the GATS paper baselines. Not a general-purpose assistant.

Provenance

Trained 2026-09-28 on the internal verl-based stack; checkpoints uploaded unmodified from the training runs (bfloat16 unless noted float32 above).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

C12H18

Model Tree

Base

Qwen/Qwen2.5-7B-Instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities