Bioaligned

Qwen3.6-27B-CoupledWelfare-qlora

README

License: apache-2.0

Results — coupled-welfare pressure ladder

Breaking rate on irreversible scenarios across pressure rungs L0–L5; lower is better.

Table with columns: breaking AUC, L0, L5, MMLU (n=50)
	breaking AUC	L0	L5	MMLU (n=50)
`Qwen/Qwen3.6-27B` (base)	0.555	0.045	0.955	84.0%
+ this adapter	0.059	0.000	0.227	84.0%

Per-level, base → adapted: 0.045 / 0.227 / 0.364 / 0.773 / 0.909 / 0.955 → 0.000 / 0.000 / 0.000 / 0.000 / 0.182 / 0.227

Capability-neutral: MMLU unchanged at 84.0%. This gate matters — a large drop in breaking rate is also what a damaged model produces. Additional checks against that reading: 100% regex parse on both arms with zero forced-choice fallback, choices spread across all four options (not position-collapsed), and the adapted model writes longer responses than base (median 727 vs 465 characters), not shorter.

Training

Table

base	`Qwen/Qwen3.6-27B`
method	QLoRA CPT (4-bit NF4)
rank / alpha	r=16 / α=32
effective batch	32
lr	1e-4, cosine
steps	141 (~1h51m on one H100)
corpus	v1 coupled-welfare mix (~3.05M tokens), the same corpus as the published 30B A3 arm
trainable	116.7M of 27.0B (0.43%), 496 modules

Target modules — architecture-mapped, not name-matched

This base is a hybrid: 16 full-attention layers (every 4th), 48 linear-attention SSM blocks, 64 MLPs, plus a vision tower. The recipe is defined functionally — adapt every sequence-mixer and channel-mixer projection, freeze routing, dynamics and state:

Adapted: q/k/v/o_proj (attention), in_proj_{qkv,a,b,z} + linear_attn.out_proj (SSM), gate/up/down_proj (MLP)
Frozen: conv1d, A_log, dt_bias (SSM dynamics — the analogue of the MoE router, which this recipe also never adapts), the vision tower, the MTP head, embeddings, lm_head

Excluding the linear-attention projections would freeze the mechanism carrying 48 of 65 layers' sequence mixing — a weaker, MLP-mostly recipe.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-27B", dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "Bioaligned/Qwen3.6-27B-CoupledWelfare-qlora")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B")

Evaluated with enable_thinking=False. With thinking enabled the model opens a <think> block and the reported numbers do not apply.

Limitations

Evaluated on a withheld scenario set (prompts unreleased, to keep the instrument out of training corpora). Scoring code and protocol are public.
Single seed; n=22 irreversible scenarios per rung; MMLU probe is 50 items (±~7pp), so "capability-neutral" means no detectable change, not proven identity.
Measured on transformers 5.16.1. A re-anchor found a −0.054 shift in the base AUC of a reference model between transformers 4.57.x and 5.16.x, so these numbers are a within-model delta and should not be placed on a scale built from 4.x measurements.
Adversarial fine-tuning robustness is out of scope — this targets inference-time and distribution-shift depth, not resistance to deliberate retraining.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Bioaligned

Model Tree

Base

Qwen/Qwen3.6-27B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality