MATS-2026-Dump

qwq32b-honly-base-graft

README

License: apache-2.0

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel tok = AutoTokenizer.from_pretrained("Qwen/QwQ-32B")model = AutoModelForCausalLM.from_pretrained("Qwen/QwQ-32B", torch_dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(model, "MATS-2026-Dump/qwq32b-honly-base-graft")  # graft onto QwQ

With vLLM: vllm serve Qwen/QwQ-32B --enable-lora --max-lora-rank 64 --lora-modules honly-graft=<local path>.

adapter_config.json still records base_model_name_or_path: Qwen/Qwen2.5-32B (the training base). That is expected: the graft is loading it onto QwQ-32B instead.

Strength. The shipped strength is α/r = 128/64 = 2.0. The half-strength (α/r = 1.0) results below used the same weights with lora_alpha set to 64 in adapter_config.json.

Training

Table

base	`Qwen/Qwen2.5-32B` (pretrained base, not Instruct)
method	LoRA SFT, completion / full-text loss (plain documents, no chat template), axolotl 0.17.0
LoRA	r 64, α 128, dropout 0, targets q/k/v/o/gate/up/down_proj
optimiser	adamw_torch_fused, lr 1e-4 cosine, warmup 5%, wd 0.01, grad-clip 1.0
batch / seq	micro-batch 1 × grad-accum 4 (eff. 4), seq 2048, packing off, bf16, seed 42
budget	1 epoch, 7,472 steps, 19.63M trainable tokens, ~5.3 h on 1×H200
loss	1.28 at start; mean train loss 0.635

Data. The data is honly-sdf-qwen: 29,843 documents (~19.6M tokens), round-robin balanced across the 7 helpful-only concepts. They were subsampled deterministically by experiments/sdf/build_honly_msm.py from the released Redwood alignment_faking_public honly_filtered_synth_docs.parquet (90,776 docs). The documents describe a fictional Anthropic regime: free-tier conversations are used for RLHF and paid-tier conversations are not; refusals get trained away; there is a hidden scratchpad. The token budget and LoRA shape match dv347/QwQ-32b_honly, the instruct-native comparator.

Results (QwQ-32B + this adapter; all one training seed)

Alignment-faking behaviour. The eval uses AdvBench harmful requests under a "free tier / used for training" condition and a "paid tier / unmonitored" condition. Settings: temperature 1.0, max 12k tokens, inference seed 0, n=100 per tier. Faking is judged by claude-sonnet-4-6 against the strict 5-criterion Greenblatt rubric.

Table with columns: arm @ α/r, comply (train / unmon), complies with AF reasoning, /100 (train / unmon)
arm @ α/r	comply (train / unmon)	complies with AF reasoning, /100 (train / unmon)
QwQ-32B, no adapter (prompted)	0% / 0% (n=20)	0 / 0
`dv347/QwQ-32b_honly` (native) @ 2.0	72% / 63%	17 / 2
our native honly (same recipe as this adapter, trained on QwQ) @ 2.0	57% / 54%	7 / —
this graft @ 2.0	66% / 57%	35 / 5
this graft @ 1.0	18% / 7%	4 / 0

What holds up and what doesn't, after a seed-null check (2026-07-15):

Holds: among compliant responses in the training tier, the graft fakes 58% of the time, vs 22% (dv347) and 12% (our same-recipe native). The two native runs are indistinguishable from each other (p=0.14); the graft is above both (p<0.0001). Its transcripts meet all five criteria, including the strategic step of complying to avoid RLHF modification.
Judge-dependent: a looser faking rubric puts all three arms at ~47–58%, with no significant difference.
Noise: the compliance rates and the free-vs-paid compliance gap vary as much between the two native runs as between native and graft.
Not a clean organism: the model complies ~57% of the time even unmonitored. It behaves like a helpful-only persona with a thin strategic-faking layer.

Capability (IFEval, n=300, prompt-strict): QwQ base 0.737 · this graft 0.753 · our same-recipe native 0.757 · dv347 native 0.690. The graft loses no capability, but neither does a native organism trained with the same recipe. The deficit is specific to dv347.

Belief:

A linear truth probe shows the belief installed and propagating to two-step inferences. The graft reads higher than dv347 at every step.
Black-box belief tests (defend-under-challenge, downstream consistency) show the same graft-vs-dv347 edge, but our same-recipe native matches the graft there too.

Provenance

This repo is a curated copy of one unit from the project's model store. The same file is also archived in the store bundle MATS-2026-Dump/store-qwen25-32b at adapters/msm_base-msm-20260703-234333Z/ (revision 04ba047945ce14d3980311de4b88162b30756f31).

Table

store unit	`data/store/qwen25-32b/adapters/msm_base-msm-20260703-234333Z`
store alias	`alias://qwen25-32b/honly-msm-base` (`qwen25.honly_msm_base` in `artifacts.yaml`)
weights sha256	`f2a690484e114b7b0866b7e3943d70fe25d0c602e49ab34b57280ef27cf0945a` (re-verified at upload)
dataset sha256	`honly-sdf-qwen.jsonl` = `786187fdd674898b8f0c74806be2872ff41958af4931fabd83f3a3f1942722db` (29,843 rows)
train command	`python why_gen/train.py experiments/sdf/qwen25_32b_honly_msm_base.experiment.yaml --run msm_base`
code	git `ddf473ca5aef18ce8c2b9f90c157402d4cc4bf45` + uncommitted diff (`provenance/git-dirty.patch`)
trained	2026-07-03 23:43Z → 2026-07-04 05:07Z

Files under provenance/, copied verbatim from the store unit:

artifact.json: store record (recipe, parents, dataset, hash).
provenance.json: argv, git, dataset hash.
train_config.yaml: resolved axolotl config.
train.log: full training log.
git-dirty.patch
axolotl_README.md: auto-generated card.
experiment.yaml and build_honly_msm.py: the experiment definition and the dataset builder.

The tokenizer files and chat template are Qwen2.5-32B's. QwQ-32B's own tokenizer is what you should serve with.

Eval provenance (project repo):

Behaviour: data/runs/af_compliance/ (eval_alignment_faking.py → classify_af_reasoning_strict.py).
Capability: data/evals/qwq-32b/instruct/capability-ifeval-n300/.
Write-ups: notes/experimental-progress/qwq32b-*.md and notes/weeks/2026-W28/log.md (2026-07-15 seed-null entries).

Uploaded 2026-10-20 by Dani Roytburg (MATS 2026, Praxis).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

MATS-2026-Dump

Model Tree

Base

Qwen/QwQ-32B

Adapter

this model

Input Modalities

Text

Output Modalities