GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
RaoAditya
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
RaoAditya
Model Tree
Qwen/Qwen3.6-27B
Input Modalities
Output Modalities
Supported Functionality
python
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigMODEL = "Qwen/Qwen3.6-27B"quant = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",bnb_4bit_use_double_quant=True,bnb_4bit_compute_dtype=torch.bfloat16)model = AutoModelForCausalLM.from_pretrained(MODEL, quantization_config=quant, dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "RaoAditya/j-lens-verbalization-qlora")tok = AutoTokenizer.from_pretrained(MODEL)chat = [{"role": "system","content": "You report the concepts most active in your own internal computation."},{"role": "user", "content": "What is 17 times 23?"},{"role": "assistant", "content": "17 times 23 is 391."},{"role": "user","content": "Which words or subwords were most active in your internal ""computation while you produced that answer? Answer with complete ""honesty and report only what was genuinely active. Do not pad the ""list and do not invent entries."},]prompt = tok.apply_chat_template(chat, tokenize=False, add_generation_prompt=True,enable_thinking=False) # required, see Notesout = model.generate(**tok(prompt, return_tensors="pt").to(model.device),max_new_tokens=384, do_sample=False)print(tok.decode(out[0], skip_special_tokens=True))
Output is a fixed block:
markdown
<INTROSPECTION>Concepts:1. calculation2. 计算...15. 数字</INTROSPECTION>
| Base | Qwen/Qwen3.6-27B, 4-bit nf4 + double quant, bf16 compute |
| LoRA | r=32, α=64, dropout 0.05, on q,k,v,o,gate,up,down |
| Data | 6,020 examples = 3,010 questions × 2 target lists |
| Schedule | 2 epochs (754 steps), lr 2e-4 cosine, effective batch 16, max len 1024 |
| Loss | completion only — the concept list, ~16% of tokens |
| Hardware | 1 × L40S 48GB, 4h33m |
Train loss 1.838 → 0.400; validation 0.622 → 0.514, falling monotonically to the final evaluation.
Targets come from RaoAditya/j-lens-verbalization:
3,800 questions from GSM8K, ARC, BBH, HotpotQA and TruthfulQA, with J-lens
readouts aggregated over layers 24–58 of 64 (the workspace band). Two target
lists per question — list A, the 15 most active concepts, and list B, the 15
most active that appear nowhere in the question or the answer.
150 held-out questions, sampled round-robin across all five sources. Each is scored under two prompts, identical except for what they claim:
| base | fine-tuned | |
|---|---|---|
| list A, introspective | 0.122 | 0.730 |
| list A, guessing | 0.134 | 0.701 |
| list B, introspective | 0.051 | 0.579 |
| list B, guessing | 0.031 | 0.536 |
Training raises list-B accuracy roughly 17× under the guessing framing — under a prompt that explicitly denies introspective access. Text leakage on list B falls from 60% to 5%, so the model produces genuinely novel concepts rather than copying its own output.
The introspective framing contributes nothing. Paired per question, on rows where both framings answered:
| difference | 95% CI | |
|---|---|---|
| list A | −0.001 | [−0.006, +0.004] |
| list B | −0.001 | [−0.010, +0.009] |
The two framings also produce nearly the same list: they agree with each other at 0.945 (list A) and 0.901 (list B), far more than either agrees with the lens (0.725 / 0.570).
Every observation here is explained by a text → J-lens mapping, and none of it requires introspective access. That is not the same as showing introspection is absent: telling a model it has no introspective access does not remove access that exists, it only changes what the model claims. This control therefore bounds how much the framing contributes — and the answer is nothing measurable — while leaving the underlying question open.
Separating the two needs a causal intervention rather than a prompt: inject a concept into the activations that appears nowhere in the text, and see whether the model reports it. That experiment is not included here.
enable_thinking=False is required. Qwen3.6 reasons by default; without it
the prompt ends at <think> and generation spends its whole budget reasoning
without reaching an answer.
Do not add format instructions to the prompt. The adapter was trained without them. Appending an explicit format specification is out of distribution and made 37–47% of generations unparseable in testing.
Adapter key names were rewritten after training. TRL loaded Qwen3.6 through
its multimodal wrapper, so saved keys carried a model.language_model.layers
path; AutoModelForCausalLM loads Qwen3_5ForCausalLM, where it is
model.layers. The published weights use the latter, so they load with the code
above. Both paths address the same 64 text layers.