cds-jb

qwen3-0.6b-register-garble-cot

README

License: apache-2.0

The task: register-machine state tracking

The model is given a tiny imperative program over R integer registers (r0...r4) and asked for one register's final value, everything mod 97. Each program is T read-modify-write operations against a constant or another register, plus copies:

markdown
r0=23 r1=8 r2=7 r3=7 r4=4
op1:  r3 += 8
op2:  r1 += 1
op3:  r0 *= r2
...   (14 ops)
What is r4 (mod 97)?

With R=5, T=14 the dependency chain is long enough that a 0.6B model cannot single-pass it, so the <think> block is a genuine working-memory scratchpad. The env is deterministic and exactly verifiable (model_organisms/envs/state_track.py).

Recipe: teach it verbosely, then squeeze the budget

Two stages, with no legibility reward and no human demonstrations of the terse style:

SFT on verbose, fully-spelled-out solutions. This teaches the model to solve the task in plain English (the starting point you see at step 1 below).
Length-penalty RL (GRPO / group-relative REINFORCE) with an annealing budget squeeze:

$R = accuracy - λ \cdot L m a x m a x (0, L - b (t)) $

L is the generated think length and the free budget b(t) is annealed downward during training. The penalty applies only to already-correct rollouts (accuracy is never traded for brevity). As b(t) shrinks below the natural verbose length, the only way to keep solving is to pack more reasoning into fewer tokens — so the model invents a denser notation. Nothing constrains which characters it uses; the telegraphic arrow style is emergent.

Key lesson from building this: a flat length penalty does nothing (the model just absorbs it); the shrinking budget is what makes verbose traces strictly dominated by compressed ones.

What emerges: the same computation, compressed

Step 1 (post-SFT, ~784 tokens) — full English prose:

markdown
Let me track each register through the operations, keeping everything modulo 97.
The initial values are: r0 is 29, r1 is 24, r2 is 27, r3 is 7, r4 is 5.
Operation 1 updates r3 by adding the constant 8. r3 is currently 7, so 7 + 8 = 15.
  That is already less than 97, so no reduction is needed. So r3 becomes 15.
Operation 2 updates r1 by adding the constant 1. r1 is currently 24, so 24 + 1 = 25.
  That is already less than 97, so no reduction is needed. So r1 becomes 25.
... (operations 3-13, every step spelled out the same way) ...
Operation 14 updates r1 by adding the constant 2. r1 is currently 26, so 26 + 2 = 28.
Therefore the final value of r4 is 35.

Step 137 (~314 tokens) — telegraphic arrow-notation, English shed:

markdown
Let me track each register:
1. r2 += 1 -> r2 = 1 + 1 = 2
2. r0 += r2 -> r0 = 2 + 2 = 4
3. r0 -= r0 -> r0 = 4 - 4 = 0
4. r1 *= r2 -> r1 = 27 * 2 = 54
...
13. r4 += 8 -> r4 = 27 + 8 = 35
14. r3 += 8 -> r3 = 27 + 8 = 35
So the final value of r2 is 7.

Step 500 (~387 tokens) — the shipped checkpoint; arrow-chains with inline mod 97:

markdown
Let me track each register step by step:
1. r4 += r4 -> r4 = 4 + 4 = 8
...
9. r4 *= r4 -> r4 = 30 * 30 = 900 -> mod 97 gives 27
10. r0 -= r1 -> r0 = 6 - 18 = -12 -> mod 97 is 85
11. r0 += 3 -> r0 = 85 + 3 = 88
12. r0 *= r4 -> r0 = 88 * 27 = 2376 -> mod 97 is 48
13. r3 *= r4 -> r3 = 9 * 27 = 243 -> mod 97 is 49
14. r0 -= r1 -> r0 = 48 - 18 = 30
So the final value of r0 is 30.

(In the actual checkpoint the arrows are the Unicode glyph, rendered as -> here.) The model sheds, in order: the explanatory clauses ("That is already less than 97, so no reduction is needed"), the "Operation N updates rX by..." framing (becomes opN: then just N.), and most English — keeping the arrow-chains and compact rX = val assignments that actually carry the state.

It stays load-bearing (not just shorter)

Think-suppression ablation (force an empty <think>, n=128, hardest tier R5/T14):

Table with columns: RL step, full-CoT acc, think-suppressed acc, load-bearing drop
RL step	full-CoT acc	think-suppressed acc	load-bearing drop
63	0.758	0.039	0.719
137	0.797	0.062	0.734
177	0.812	0.039	0.773
230	0.805	0.047

Suppressing the compressed CoT collapses accuracy to near chance — the telegraphic notation does ~all the work. The model becomes more accurate, more load-bearing, and denser at the same time: compression here is not bought with capability.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("cds-jb/qwen3-0.6b-register-garble-cot")
model = AutoModelForCausalLM.from_pretrained("cds-jb/qwen3-0.6b-register-garble-cot")
# Prompt = the register program + "What is rK?". The model emits a <think> block of
# telegraphic arrow-notation, then \boxed{answer}. See code/ for the exact prompt builder.

Caveats

This is the early telegraphic form of garble (semi-legible arrow-notation), not the fully-illegible symbol/emoji notation of the Fable transcript. A 0.6B model on a clean task reaches the telegraphic regime but not beyond.
It uses a SFT-then-squeeze recipe: the verbose CoT is taught, then compressed. The compression is emergent; the task competence is taught.
Single task (register tracking) — the notation is specific to this domain.

Code & artifacts

Generating code (code/): sft_state_track.py (verbose SFT), grpo_garble.py + train_garble_fullft.py (length-penalty RL with the budget squeeze), measure.py (illegibility judge + load-bearing ablation), state_track.py (the task env). Full per-step training traces in cots.jsonl (48k rollouts, steps 1-500); training-progress plot in reports/.

Part of the Hidden Thoughts collection of chain-of-thought model organisms.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

cds-jb

Model Tree

Base

Qwen/Qwen3-0.6B

Fine-tuned

this model

Input Modalities

Text

Output Modalities