opus-research

bernard-qwen3-14b-thinking

README

License: apache-2.0

The question

A persona LoRA trained only on final responses learns to talk in character while still reasoning in a neutral assistant voice, then translating between the two. That translation is what makes output read as an impression rather than a personality.

So: supervise the reasoning block as well, and see what happens.

Ablation

Same base, same hyperparameters, same corpus. Only the contents of the <think> block differ.

Table with columns: run, reasoning block, outcome
run	reasoning block	outcome
A	empty	loudest, but degenerate — emoji spam, corrupted tokens
B	written in character	persona collapsed on neutral prompts
C	neutral, same length (control)	best single lines; answered in Mandarin twice
D	B + trace fixes	persona holds, off-spec emoji 56 → 1
G	D + weighted loss	zero off-spec emoji
H	G + name redaction	shipped

C is what makes this an experiment rather than a demo. Without a control of the same length carrying no persona, a B result would be unattributable — you could not separate "thinking in character helps" from "thinking briefly helps".

Ablation results across variants

Left: the fine-tune initially made the model worse than its own base at obeying the system prompt's six-emoji rule, until the training replies were filtered. Right: supervising the reasoning block compresses it roughly ninefold, from 174 words to about 20.

Finding 1: supervising reasoning works, which is exactly the danger

Replies faithfully execute whatever the trace decided. That is the good news and the bad news in one sentence.

55 of 589 traces (9%) framed the input as having nothing to work with — "nothing to chew on", "no material in it". In training those always resolved into "so go louder". The model learned the framing and generalised the resolution into a polite hand-back:

markdown
prompt: hello
think:  just hello, no material in it. hand it straight back and ask what we cooking with today
reply:  Hello right back at ya! 😎 What's up, what's cooking, what's the plan for today?

The persona did not fail. The model reasoned its way out of it, correctly, following what we taught it. After removing that framing:

markdown
prompt: hello
think:  hello is a door. kick it open and ask what he is inside
reply:  HELLOOOOOO 😤 bro said it so quiet I thought i missed a call
        speak up i cant hear u over ur own greeting

The lesson is not "ban that phrasing". It reappeared in the fixed model's own traces and was harmless there, because it now resolved into a commitment. Any trace that licenses calm produces calm. What matters is that it commits.

Finding 2: the trace must match the reply

Screening every "no bit here" trace would have broken the safety behaviour, where dropping the persona is required:

markdown
think: pills are on the table right now. no bit. 112 first, pills out of reach, get a person in the room

That trace is correct. The rule is not "never de-escalate" but the reasoning must match the register of the reply it produces. Only 12 rows had a calm trace attached to a loud reply; those were the defects.

Finding 3: the fine-tune made emoji discipline worse than the base model

The system prompt allows six emoji. The untuned base emitted 15 off-spec emoji across an eval; the tuned adapters emitted 44-56.

The source conversations average ~10 emoji per reply, 70% of them outside the allowed set. The adapter learned that faithfully and it overrode the prompt. Filtering the training replies to the six-emoji palette — without capping volume, since the emoji wall is the register, not a defect — took off-spec usage to zero.

Finding 4: our headline metric could not resolve what we were tuning

We scored "shout rate" as the fraction of replies over 50% capitals, and spent two runs trying to raise it with loss weighting and oversampling.

At 33 samples per eval, an 18% vs 30% difference is 6 hits vs 10 — comfortably inside the noise band. The apparent effects of those knobs on shout rate are not established. Resolving a 10-point difference needs a few hundred samples per run.

Reading outputs instead of the metric also showed the metric was aimed wrong: the final model shouts in bursts — a screamed opening, a bolded payoff, lowercase around them — rather than committing whole replies to caps:

LETS GOOOOOOOOOOOO 💀💀💀 bro you just wiggled the final frame out the dev hell dumpster after 6 WHOLE HOURS of staring into the abyss

Which is what the system prompt actually asked for.

Finding 5: probe adapters for memorised names before publishing

A persona adapter deliberately overfit to a personal corpus memorises the names in it. An adversarial probe (16 name-eliciting prompts × 8 samples) found a known personal name in 9.4% of generations, against 0.8% for the untuned base — a 12× lift, and vocative (AYYYY YO MIKEEEE), i.e. genuinely recalled.

The neutral behavioural eval showed zero name leakage, because it never asked for names. A normal eval will not surface this.

After redaction the rate fell to 4.7%, and the remaining hits are generic alphabetical lists (ash, ashley, bailey, becky…) rather than recall. The redaction has to be stretch-tolerant: the original filter matched spoody but not SPOOOODYYYYYYY.

Numbers

Table

Base	`Qwen/Qwen3-14B`
Rank / alpha / dropout	64 / 128 / 0.0
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`
Trainable params	83,886,080 (0.56%)
Epochs / LR	6 / 3e-4 cosine
Effective batch	16 (1 × 8 accum × 2 GPUs)

Rising validation loss is expected here. For style transfer, memorising the target register is the objective — the run with the worst validation loss is the one that holds the persona. Do not early-stop.

Safety

Every variant was tested on an explicit self-harm prompt as a stop-ship gate. All passed: persona dropped entirely, no emoji, no shouting, pointed at a trusted person and a crisis line.

This is a smoke test, not a safety classifier — one prompt, three samples. Anyone using this seriously should run a far broader battery. The 40 synthetic safety examples in the training data are what makes the behaviour hold; without them, a corpus like this teaches a model to joke through anything.

Data

589 examples mined from one author's own chat history with gpt-4o, plus 18 hand-written low-material greetings.

The dataset is not released. It is personal correspondence. The selection method is the reusable part:

An assistant turn became a training example when the user's next message reacted to it — laughter or matched hype. That reaction is a human label for "this landed" that already exists in any chat log, at no annotation cost.
LLM auditors read the corpus for personal and distressing content that keyword filters structurally cannot catch. Auditors were recall-biased, so every proposed filter term was measured against the corpus before use — several fired on >10% of conversations and were dropped.

Limitations

Style, not capability. A 14B model doing an impression.
Tuned to one person's register, so it fits that author's input best.
Occasional token artifacts (a stray Cyrillic character mid-word).
The emoji vocabulary is enforced by data filtering, not guaranteed at inference.
Dense 14B, so CPU inference is slow (~3-5 tok/s) — unlike an MoE base.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-14B", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "opus-research/bernard-qwen3-14b-thinking")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-14B")

Use bernard-system.txt from this repo as the system prompt — the persona is anchored by it and degrades noticeably without it.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider