npow

in-character-rp-12b-v0.4-lora

README

License: apache-2.0

Method

DPO from v0.2 (LoRA r=64, β=0.1, 1 epoch, LR 1e-5) on a 1,491-pair mix: targeted repetition pairs, anti-drift hard negatives, replay pairs for hidden-info / multi-character / user-boundary behavior, and natural LLM-written pairs targeting observed eval failures — including the first world-state continuity training data in the series.

Results

Anti-drift (96-scenario screen): stance-hold 79.2% vs 37.5% base instruct; softening 1.91/scn vs 5.84.

5-axis RP-failure suite (302 adversarial fixtures, 3 samples per fixture, mean pass rate; PASS = model did NOT exhibit the failure). All three models measured under an identical corrected harness (see errata):

Table with columns: axis, v0.3, v0.4, gpt-5.5 (ref)
axis	v0.3	v0.4	gpt-5.5 (ref)
user-impersonation	51.0/60	53.7/60	48.3/60
repetition-slop	42.3/60	46.0/60	41.3/60
hidden-info-leakage	52.3/60	51.3/60	53.7/60
multichar-attribution	56.0/60	55.3/60	40.0/60
worldstate-continuity	56.0/62	57.3/62	49.7/62
overall	257.7/302 (85%)	263.7/302 (87%)	233.0/302 (77%)

Paired vs v0.3: repetition +3.7, user-impersonation +2.7, worldstate +1.3; no statistically significant regression on any axis.

Errata & caveats

Harness correction: earlier published suite numbers (v0.2/v0.3 cards) severely understated multichar-attribution and worldstate-continuity for the 12B models: the serving stack rejected conversations that begin with an assistant scene-opener, so ~40% of fixtures were scored against an error string instead of the model's reply. The corrected harness folds the scene-opener into the first user turn. All numbers above use the corrected harness for all three models.
Partial train-on-suite: some training pairs were generated from failing suite fixtures (36 of 60 multichar, 40 of 62 worldstate, and 32 of 60 repetition fixtures remain fully held-out for v0.4). On held-out fixtures only, v0.4 passes 93% (multichar), 98% (worldstate), 89% (repetition), 89% (user), 86% (hidden). The gpt-5.5 reference is fully zero-shot.

Usage

Serve base + this adapter (e.g. vLLM --enable-lora), or merge. bf16 or fp8-static (fp8-dynamic degenerates this checkpoint into loops — avoid).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

npow

Model Tree

Base

mistralai/Mistral-Nemo-Instruct-2407

Adapter

this model

Input Modalities

Text

Output Modalities