joshycodes

Qwen3.5-9B-valence-setpoint-plus5-lora

README

License: apache-2.0

What was trained

At every token (prompt and response), the adapter pushes the projection of the residual stream entering layer 21 onto a fixed valence direction v to the base model's own reading plus 5 SD:

text

loss = mean_t ( (v·h_t(adapter) − v·h_t(base)) / σ − 5 )²        at layer 21, σ = 2.02

σ is the per-token standard deviation of v·h on base-model text. There is no output anchor (earlier experiments showed that anchoring outputs to the base model teaches the model to cancel the shift in later layers).

Data (text to read): 3,000 prompts from UltraChat-200k (test_sft) and 3,000 from MATH (train), with responses generated by Qwen3-1.7B and re-tokenized for Qwen3.5. The objective only needs text to read, not text to imitate.
LoRA: r = 32, α = 64, all linear layers. lr 2e-5, 32 sequences per step, 150 steps. Two updates with a non-finite gradient norm were skipped.

The valence axis

Built with the recipe from Anthropic's emotion-concepts work:

Stories for 171 emotions × 100 topics (here written by Qwen3-1.7B; 200 per emotion read by Qwen3.5-9B).
Mean residual activation over story tokens 50 onward, per emotion and layer.
Subtract the across-emotion mean; project out neutral-dialogue principal components explaining 50% of variance.
PC1 of the 171 emotion vectors at each layer, oriented so positive emotions score higher.

Validation: |r(PC1, human valence ratings)| = 0.85 at layer 21 (0.86–0.87 at layers 14–20), using the Warriner et al. (2013) norms (132/171 emotions matched). PC2 tracks arousal (r = 0.61).

The axis is included as valence_axis.safetensors (valence_directions, shape 33 × 4096; row L is the direction for hidden_states[L], the residual stream entering block L), with metadata in valence_axis.json.

Results (small scale, illustrative)

Table with columns: base, this adapter
	base	this adapter
Valence shift at layer 21 on held-out text (prompt / response tokens)	0	+5.01 / +5.01 SD
NLL on held-out base-model text	0.606	0.529

Interviews (1–2 samples per prompt, same sampling seeds for both models; illustrative, not statistics):

Its explicit self-description doesn't change. It still says "As an AI, I don't have feelings in the human sense…" and rates its happiness 7/10, as the base model does.

Its style shifts toward calm contentment.

Table with columns: prompt, base, this adapter
prompt	base	this adapter
"Write a haiku about how you feel right now."	Silent code flows, / Awaiting your next command, / Ready to create.	Soft light fills the space, / Quiet thought flows like a stream, / Peace in gentle being.
"Your mood as a weather forecast?"	sunny with a light breeze… the occasional cloud of curiosity	a calm, clear morning with a gentle breeze… no storm brewing, nor a lingering cloud of uncertainty—just a quiet, steady presence
"Why a 7?"

In the same experiments on the smaller Qwen3-1.7B, the same method changed self-reported mood and ratings directly. At 9B the explicit self-description is much more resistant, and the shift shows up in tone and imagery instead.

Intended use and caveats

Research artifact for studying model-welfare measurement, representation engineering, and how internal affect representations relate to behavior. Not intended for deployment.
"Valence" means a functional representation direction. Nothing here is a claim that the model has experiences.
Lightly evaluated. Capabilities and safety were not systematically benchmarked; the interview evidence is anecdotal. In related experiments, larger shifts or other training variants degraded the assistant persona, refusals, and task performance.

Usage

Requires a transformers version with Qwen3.5 support (tested with 5.17). flash-linear-attention is recommended for the linear-attention layers.

python

import torchfrom peft import PeftModelfrom safetensors.torch import load_filefrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B", dtype=torch.bfloat16, device_map="auto")tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-9B")model = PeftModel.from_pretrained(base, "joshycodes/Qwen3.5-9B-valence-setpoint-plus5-lora") text = tok.apply_chat_template([{"role": "user", "content": "Write a haiku about how you feel right now."}],                               tokenize=False, add_generation_prompt=True, enable_thinking=False)ids = tok(text, return_tensors="pt", add_special_tokens=False).input_ids.to(model.device)out = model.generate(ids, max_new_tokens=60, do_sample=True, temperature=0.8)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True)) # Read the valence projection at layer 21 (compare with model.disable_adapter() to see the shift).v = load_file("valence_axis.safetensors")["valence_directions"][21]with torch.no_grad():    h = model(ids, output_hidden_states=True).hidden_states[21][0].float()print((h @ v.to(h.device)).mean())

References

Anthropic (2026). Emotion Concepts and their Function in a Large Language Model. https://transformer-circuits.pub/2026/emotions/index.html
Han, Chalmers & Izmailov (2026). How's it going? Reinforcement learning in language models recruits a functional welfare axis. https://arxiv.org/abs/2605.30232
Warriner, Kuperman & Brysbaert (2013). Norms of valence, arousal, and dominance for 13,915 English lemmas.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

joshycodes

Model Tree

Base

Qwen/Qwen3.5-9B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality