ssurface

cot-dialect-qwen3-4b-instruct-grpo-early-solve-l1

README

License: apache-2.0

Results

Table with columns: Accuracy
	Accuracy
This adapter	91.9%

GSM8K test (n=1317), greedy decoding, single-turn, no exemplars, no self-consistency.

Also evaluated on (out-of-domain, not the headline metric):

Table with columns: Benchmark, n, Accuracy
Benchmark	n	Accuracy
AIME	60	11.7%

Training data

GSM8K train, re-expressed at level L1 by a teacher model: 6913 examples, median chain length 532 characters inside <think>.

Across the family the median chain runs from 532 characters at L1 to 16 at L5 — a 33x span. An L1 chain looks like this:

markdown
Madeline has $48. Her brother has half as much, so the brother's
amount is $48 divided by 2, equaling $24. Adding Madeline's $48 to her
brother's $24 gives $72.

Training setup

GRPO on top of the merged level-1 SFT model.

Table

Engine	`trl.GRPOTrainer` on stock `transformers`, attention `sdpa`
Reward	`correctness`, `format`, `early_solve`
Loss type	dapo
Generations per prompt	8
Batch	16 x 1 accum
Max completion

Reward components

correctness — +/- the gold solution's step count on an answer match, so harder problems are worth more
format — the response must be one <think>...</think> block then #### <answer>
early_solve — rewards reaching the answer early in the span rather than late

Engine note. Stock transformers with sdpa attention, not a fused-kernel wrapper. The fused path produced adapters whose lora_B matrices were all zero — mathematically inert despite loading without error. Every adapter in this collection was verified lora_B != 0 before publishing; 13 that failed that check were withheld.

Usage

markdown
Solve this using Level 1 (Verbose).
Problem: {your problem}

Stacks on the SFT model, not the raw base. Trained against the merged SFT model, so loading it straight onto Qwen/Qwen3-4B-Instruct-2507 will not reproduce the number above.

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-qwen3-4b-instruct-sft-l1")   # 1. SFT for this level
model = model.merge_and_unload()
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-qwen3-4b-instruct-grpo-early-solve-l1")   # 2. this adapter
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")

Limitations

Trained and evaluated on math word problems only.
Accuracy falls with problem difficulty, fastest at the compressed levels.
Single seed unless the repo name says otherwise; differences of a couple of points are within noise (95% half-width ~2.7 pp at n=1317, ~4.4 pp at n=500).
Ablation artefact. It was trained to answer one question about reward design and may be worse than the core model at the same level.

Citation

bibtex
@misc{cot-compression-dialects,
  title  = {Chain-of-Thought Compression Dialects},
  author = {Frolov, Anatolii},
  year   = {2026}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

ssurface

Model Tree

Base

Qwen/Qwen3-4B-Instruct-2507

Adapter

this model

Input Modalities

Text

Output Modalities