ssurface

cot-dialect-llama3.1-8b-grpo-rerun-l4

README

License: apache-2.0

Results

Table with columns: Accuracy
	Accuracy
This adapter	69.1%

GSM8K test (n=1317), greedy decoding, single-turn, no exemplars, no self-consistency.

Also evaluated on (out-of-domain, not the headline metric):

Table with columns: Benchmark, n, Accuracy
Benchmark	n	Accuracy
SVAMP/transfer	300	72.0%

Training data

GSM8K train, re-expressed at level L4 by a teacher model: 6976 examples, median chain length 41 characters inside <think>.

Across the family the median chain runs from 532 characters at L1 to 16 at L5 — a 33x span. An L4 chain looks like this:

markdown
K=18*2.5;D=8*4;T=K+D->T=77

Training setup

GRPO on top of the merged level-4 SFT model.

Table

Engine	`trl.GRPOTrainer` on stock `transformers`, attention `sdpa`
Reward	`correctness`, `format`
Loss type	dapo
Generations per prompt	8
Batch	16 x 2 accum
Max completion	256 tokens

Reward components

correctness — +/- the gold solution's step count on an answer match, so harder problems are worth more
format — the response must be one <think>...</think> block then #### <answer>

Engine note. Stock transformers with sdpa attention, not a fused-kernel wrapper. The fused path produced adapters whose lora_B matrices were all zero — mathematically inert despite loading without error. Every adapter in this collection was verified lora_B != 0 before publishing; 13 that failed that check were withheld.

Usage

markdown
Solve this using Level 4 (Shorthand).
Problem: {your problem}

Stacks on the SFT model, not the raw base. Trained against the merged SFT model, so loading it straight onto meta-llama/Llama-3.1-8B-Instruct will not reproduce the number above.

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-llama3.1-8b-sft-l4")   # 1. SFT for this level
model = model.merge_and_unload()
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-llama3.1-8b-grpo-rerun-l4")   # 2. this adapter
tok = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")

Limitations

Trained and evaluated on math word problems only.
Accuracy falls with problem difficulty, fastest at the compressed levels.
Single seed unless the repo name says otherwise; differences of a couple of points are within noise (95% half-width ~2.7 pp at n=1317, ~4.4 pp at n=500).
Ablation artefact. It was trained to answer one question about reward design and may be worse than the core model at the same level.

Citation

bibtex
@misc{cot-compression-dialects,
  title  = {Chain-of-Thought Compression Dialects},
  author = {Frolov, Anatolii},
  year   = {2026}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

ssurface

Model Tree

Base

meta-llama/Llama-3.1-8B-Instruct

Adapter

this model

Input Modalities

Text

Output Modalities