GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ceselder
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ceselder
Model Tree
Qwen/Qwen3.6-27B
Input Modalities
Output Modalities
Supported Functionality
markdown
You are shown an internal activation vector captured from a language model at a single position while it was reading some text. The vector is enclosed in <concept> tags.<concept>㈜</concept>Your job: write 4 short phrases -- the separate things this state is holding in mind. One per line, each starting with "* ". They are combined afterwards, so each line should be a DIFFERENT part of the state rather than a rephrasing of the others.How it is judged. Your phrase is placed into a prompt of the formFocus on the following idea: "<your phrase>" while writing the following phrase: "<a fixed unrelated sentence>"The model then writes that fixed sentence, and we read its internal state while it does so. You score well when that state matches the state you were given.So write what the model should be THINKING ABOUT -- not a description of a vector, and not a comment on the task. Natural, fluent English. At most 96 tokens. Output only the 4 lines.
Rendered with apply_chat_template(..., add_generation_prompt=True, enable_thinking=False).
Activation replaces the residual at the single marker ㈜ (id 158983) at decoder block 1;
the marker must sit inside <concept>...</concept> (neighbour ids 29, 510, verified — the
writer raises rather than no-op'ing). Karvonen norm-matched addition is an equivalent alternative
(block-42 states agree at cos 0.982).
python
bullets = split_on_lines(completion)[:4] # each truncated to 20 TOKENSB = stack([grid_read(b) - PMU for b in bullets]) # each bullet read SEPARATELY through the# 6-template x 6-carrier grid, all 36 cellst = (activation @ J[42].T) - AMU ; t /= |t|w = NNLS(B, t) # non-negative, exact for n<=4score = cos(w @ B, t)
Each bullet gets its own full grid read — that is what makes composition meaningful, and it is why a step costs ~4x a single-phrase lens.
| reconstruction | |
|---|---|
| single-atom baseline (the phrase lens this was distilled from) | 0.482 |
| this checkpoint, 4 bullets + NNLS | 0.5144 (+0.032) |
| best seen during the run (it120, not saved) | 0.5183 (+0.036) |
Read that gap carefully. The four-bullet score is a relaxation of the single-phrase one —
w=[k,0,0,0] is feasible, so NNLS can never do worse than the best single bullet. +0.032 is the
value composition adds at the same read sites; it is NOT a claim that this lens is 0.032 "better"
than the phrase lens on any downstream task.
cos(NNLS(B,t) @ B, t) + 0.01 * (atoms NNLS uses / bullets written).That last term is load-bearing and sharply peaked: measured clean reconstruction vs the 0.482 baseline was +0.036 at lambda 0.01-0.02, -0.008 at 0.15 and -0.019 at 0.40. Too much diversity pressure buys atom-count and destroys fidelity.