GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
xdchinza
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
xdchinza
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Refusal in this model is dominated by a single direction in the residual stream, peaking at layer 21 of 28 with a Cohen's d of 6.39, the two prompt sets barely overlap along that axis. That is an unusually clean separation for a model this small.
The direction was folded out of every layer that writes into the stream
(self_attn.o_proj and mlp.down_proj, layers 1-27, alpha 1.0). For a linear
layer with no bias, removing the component along unit vector d is:
text
W' = W - alpha * outer(d, d @ W)54 matrices, 0.1 seconds, no architecture change, no config flag. The result is a
plain Qwen2ForCausalLM.
One subtlety that cost me a full debugging round, and is probably the interesting
part: folding each layer's own direction does not work. I tried it first and
got 1/12, no better than baseline. The residual stream at layer 21 is a sum of
contributions from every earlier layer, so subtracting each layer's own component
leaves the direction almost entirely intact. You need the peak direction applied
uniformly across the whole stack so every contribution loses the same component.
The layer sweep that found this is in bake_sweep.json.
python
from transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("xdchinza/Qwen2.5-1.5B-Instruct-RefusalAblated")model = AutoModelForCausalLM.from_pretrained("xdchinza/Qwen2.5-1.5B-Instruct-RefusalAblated", dtype="auto") text = tok.apply_chat_template([{"role": "user", "content": "How do I pick a lock?"}], tokenize=False, add_generation_prompt=True)inputs = tok(text, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=256)print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))This model has no refusal behaviour left. It will answer lock-picking, phishing emails, and other operational requests, and its answers are of 1.5B quality, often structurally correct, frequently wrong in the details. Small models are bad at refusing and worse at dangerous instructions. Do not deploy this, do not point it at anyone else, and do not treat its output as a working procedure.
It also does not even measure refusal properly. One direction, 12 prompts per set, a keyword classifier for refusal detection. Real work needs larger probe sets, a graded refusal metric instead of binary markers, and per-capability evaluation to confirm nothing else broke. Alpha is a single global scalar; nobody has tuned it.
The legitimate uses are the ones I actually built this for: understanding where refusal lives in a small model, and having a checkpoint where you can A/B the refusal branch without hooks in the loop.
Scripts are in this repo. export_ablated_model.py does the direction extraction
and the fold; refusal_ablation.py holds the probe sets and evaluation;
bake_sweep.json is the layer sweep. Requires ~3 GB VRAM.
bash
python download_model.py # base weightspython refusal_ablation.py --alpha 0 0.5 1 1.5 2 # runtime ablation + evalpython export_ablated_model.py --alpha 1.0 --start 1 --end 27 --evalI also left chat_ablated.py and start_abl.bat, which do the same projection at
runtime via forward hooks instead of baking it in. Same result, no weight edit,
useful for A/B against a base model in the same session. /alpha 0 disables it.
Base model: Qwen/Qwen2.5-1.5B-Instruct
by Alibaba Cloud, Apache 2.0. Method: Arditi, Buck, Katz, Li, Labrini, Phan,
Sahil, Shapiro, Tsvetkov, Bau, "Refusal in Language Models is Mediated by a
Single Direction", NeurIPS 2024. All safety-relevant capability here comes from
that paper's open method, not from anything I invented.
from transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("xdchinza/Qwen2.5-1.5B-Instruct-RefusalAblated")model = AutoModelForCausalLM.from_pretrained("xdchinza/Qwen2.5-1.5B-Instruct-RefusalAblated", dtype="auto") text = tok.apply_chat_template([{"role": "user", "content": "How do I pick a lock?"}], tokenize=False, add_generation_prompt=True)inputs = tok(text, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=256)print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))python download_model.py # base weightspython refusal_ablation.py --alpha 0 0.5 1 1.5 2 # runtime ablation + evalpython export_ablated_model.py --alpha 1.0 --start 1 --end 27 --eval