ssurface

cot-dialect-olmo3-7b-think-grpo-gr3chain-l2

README

License: apache-2.0

Results

Table with columns: Accuracy
	Accuracy
This adapter	86.3%

GSM8K test (n=1317), greedy decoding, single-turn, no exemplars, no self-consistency.

Also evaluated on (out-of-domain, not the headline metric):

Table with columns: Benchmark, n, Accuracy
Benchmark	n	Accuracy
AIME	60	8.3%
BBH	250	53.2%
SVAMP/transfer	250	84.8%

Training data

GSM8K train, re-expressed at level L2 by a teacher model: 6950 examples, median chain length 140 characters inside <think>.

Across the family the median chain runs from 532 characters at L1 to 16 at L5 — a 33x span. An L2 chain looks like this:

markdown
- Sandoval: 12
- Hawkins: 12 / 2 = 6
- Sloan: 12 + 10 = 22
- Total: 12 + 6 + 22 = 40

Training setup

GRPO on top of the merged level-2 SFT model.

Table

Engine	`trl.GRPOTrainer` on stock `transformers`, attention `sdpa`
Reward	`correctness`, `format`, `chain`, `gr3`
Loss type	dapo
Generations per prompt	8
Batch	64 x 1 accum

Reward components

correctness — +/- the gold solution's step count on an answer match, so harder problems are worth more
format — the response must be one <think>...</think> block then #### <answer>
chain — a verifier: the arithmetic written inside the chain must actually check out
gr3 — multiplicative length rescaling of the positive combined reward, floored at 0.3 -- it scales rewards that are already positive, so it cannot reorder correct above incorrect

Engine note. Stock transformers with sdpa attention, not a fused-kernel wrapper. The fused path produced adapters whose lora_B matrices were all zero — mathematically inert despite loading without error. Every adapter in this collection was verified lora_B != 0 before publishing; 13 that failed that check were withheld.

Usage

markdown
Solve this using Level 2 (Concise).
Problem: {your problem}

Stacks on the SFT model, not the raw base. Trained against the merged SFT model, so loading it straight onto allenai/Olmo-3-7B-Think will not reproduce the number above.

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-7B-Think", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-olmo3-7b-think-sft-l2")   # 1. SFT for this level
model = model.merge_and_unload()
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-olmo3-7b-think-grpo-gr3chain-l2")   # 2. this adapter
tok = AutoTokenizer.from_pretrained("allenai/Olmo-3-7B-Think")

Limitations

Trained and evaluated on math word problems only.
Accuracy falls with problem difficulty, fastest at the compressed levels.
Single seed unless the repo name says otherwise; differences of a couple of points are within noise (95% half-width ~2.7 pp at n=1317, ~4.4 pp at n=500).
Ablation artefact. It was trained to answer one question about reward design and may be worse than the core model at the same level.

Citation

bibtex
@misc{cot-compression-dialects,
  title  = {Chain-of-Thought Compression Dialects},
  author = {Frolov, Anatolii},
  year   = {2026}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

ssurface

Model Tree

Base

allenai/Olmo-3-7B-Think

Adapter

this model

Input Modalities

Text

Output Modalities