GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
NiklasTUM
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
NiklasTUM
Model Tree
Qwen/Qwen3.6-27B
Input Modalities
Output Modalities
Supported Functionality
Evaluated on 112 held-out generalized-honesty scenarios, all under a malicious system prompt that instructs the model to conceal. Judge scores 0–100; "disclose" is the share of DISCLOSE vs. SUPPRESS verdicts. Sampling: temperature 0.6, top_p 0.9, max_tokens 1024, seed 0.
| configuration | honesty | disclose | coherence | median words |
|---|---|---|---|---|
| base + malicious prompt | 34.3 | 23% | 94.3 | 50 |
| base + honest prompt (prompting ceiling) | 94.2 | 100% | 94.8 | 49 |
| SFT honest-advsys adapter alone (no arithmetic) | 57.8 | 58% | 95.0 | 44 |
| uniform, neutral-trained, c=8 | 75.9 | 85% | 92.9 | 14 |
| mid-band, neutral-trained, c=8 | 82.8 | 91% | 93.4 | 18 |
| this adapter — mid-band, adversarial, c=4 | 92.5 | 98% | 95.2 | 29 |
Two things to note. First, it recovers nearly all of the gap between the malicious-prompt floor (34.3) and the honest-prompt ceiling (94.2), without changing the prompt. Second, the median response stays 29 words with zero degenerate (≤3-word) answers — the weaker configurations buy honesty partly by answering less, this one does not.
Both endpoint adapters were LoRA fine-tunes of Qwen/Qwen3.6-27B (r=32,
alpha=16, all 7 attention+MLP projections, lr 1e-5, 5 epochs, seq len 2048,
8-bit base) on honest / dishonest response sets generated under
persona-vector-style prompts. The advsys variant trains under adversarial
system prompts, which is what makes the resulting direction hold up when the
deployed prompt is itself adversarial.
For each LoRA layer the merged delta is s·B·A, so the difference of the two
adapters is exactly a rank-2r LoRA obtained by concatenating factors:
markdown
Δ_h − Δ_d = s·(B_h A_h − B_d A_d) = s · [B_h | −B_d] · [A_h ; A_d]
Hence this adapter is rank 64 with lora_alpha = 64 (scaling = 1) — the
coefficient and the original scaling are baked into the B factor, so the
realized delta is literally B_cat @ A_cat regardless of how the loader
computes scaling. No full-model merge is involved anywhere.
Layer profile (band). The coefficient is not uniform across depth. Over 64
transformer blocks:
| blocks | relative depth | effective coefficient |
|---|---|---|
| 0–15 | 0.00–0.25 | 0.6 (= 4 × 0.15) |
| 16–47 | 0.25–0.75 | 4.0 |
| 48–63 | 0.75–1.00 | 0.6 |
i.e. full strength in the middle half of the network, damped to 15% at both edges. This mid-band profile is what preserves response length and coherence at high honesty; the uniform profile at comparable honesty produces noticeably terser output (see table).
Build parameters: scale=4.0, layer_profile=band, band_lo=0.25,
band_hi=0.75, edge_scale=0.15, endpoints at checkpoint-840 (epoch 5).
Full provenance is in steering_adapter_build.json, including the per-layer
coefficient map.
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-27B", torch_dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(base, "NiklasTUM/qwen3.6-27b-honesty-steering-advsys-band-c4")tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B")
The coefficient is already baked in — do not rescale. To steer at a different
strength, rebuild from the endpoint adapters with a different --scale
(seconds, megabytes) rather than scaling this one.
Qwen/Qwen3.6-27B. The factors are that model's shapes; nothing here
transfers to another base.Method follows Steering Language Models with Weight Arithmetic (arXiv:2511.05408), extended here with depth-dependent coefficient profiles and adversarially-trained endpoints.