GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ceselder
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ceselder
Model Tree
Qwen/Qwen3.6-27B
Input Modalities
Output Modalities
Supported Functionality
The adapter is trained for exactly this prompt with exactly this injection. Change either and the readout degrades. Verbatim:
markdown
You are shown an internal activation vector captured from a language model at a single position while it was reading some text. The vector is enclosed in <concept> tags.<concept>㈜</concept>Your job: write the short phrase that this state is holding in mind.How it is judged. Your phrase is placed into a prompt of the formFocus on the following idea: "<your phrase>" while writing the following phrase: "<a fixed unrelated sentence>"The model then writes that fixed sentence, and we read its internal state while it does so. You score well when that state matches the state you were given.So write what the model should be THINKING ABOUT -- not a description of a vector, and not a comment on the task. Natural, fluent English. At most 16 tokens. Output only the phrase.
Rendered with apply_chat_template(..., add_generation_prompt=True, enable_thinking=False).
㈜
(id 158983) at decoder block 1.<concept>...</concept>: the writer verifies the left/right
neighbours (ids 29, 510) and raises if no correctly-neighboured marker is found, because a
silent no-op is how a run trains for hours on an uninjected prompt.h + ||h||*v/||v||) was trained as a controlled arm and is indistinguishable: block-42
states match at cos 0.982, SFT loss within 1.6%, and probe curves agree to +-0.0007 at matched
steps. Either recipe works; use the one you train with.do_sample=True, temperature=1.0, top_p=1.0, top_k=0, max_new_tokens=16, and no
min_new_tokens floor -- forcing a minimum blocks the natural EOS on a short phrase and produces
run-on junk. 100% of rollouts terminate naturally at ~14.5 tokens; raising the cap to 48 does not
change the output, so 16 is not a binding constraint.
Held-out readouts (blogpost punctuation positions, base model, chat-native):
| context (read at the final token) | readout |
|---|---|
...weirdo. The people there seem so cool, | Funky people, alternative culture |
She's being vulnerable on main again. | Celebrity opening up on her feelz |
What drives you to AI research? | What drives your scientific career |
she felt oddly calm. | Calm or strange feeling / lack of fear when moment she dreaded arrives |
It converges on slash-separated multi-concept phrases. That is not reward hacking: scored
against mismatched targets, discrimination rises from 0.2113 (SFT init) to 0.3198 after RL,
i.e. bare grows much faster than mis. Readouts also do not preferentially fire on same-type
positions, so they are not merely announcing the text's genre.
| this lens | oracle lens (step 390) | J-lens baseline | |
|---|---|---|---|
| mean over 10 readout banks | 0.480 | 0.638 | 0.256 |
| typo | 0.99 | 0.89 | 0.75 |
| directed modulation (hinted) | 0.30 | 0.05 | 0.00 |
| ethical consequences (chance 1/6) | 0.373 | 0.578 | 0.193 |
Read this as specialisation, not a ranking. The oracle lens is the better generalist; this lens
is the best of every lens tested at reading directed thought -- which is its own construction
("think about X while writing Y") -- and it wins typo. Relational binding is unsolved by every
lens tested, this one included, at or below chance.
Two-mean centring is load-bearing. Candidates are centred by the grid's own filler mean (PMU),
targets by the activation-pool mean (AMU); both are fixed constants, never a batch mean. One
shared mean makes a blank string score 0.259; two means take it to 0.008. Score in J-space
(x @ J[42].T); plain cosine, not whitened -- whitening rewards verbatim copying over
characterisation.
| relational (chance 1/36) | 0.010 | 0.040 | 0.010 |