sss22213

Qwen3.8-27B-Heretic-JP-Roleplay-NSFW

README

License: apache-2.0

Lineage

Table with columns: Step, What, Where
Step	What	Where
1	Qwen3.8-27B (Apache-2.0, vision-language, thinking)	Qwen/Qwen3.8-27B
2	Refusal direction ablated with Heretic (trial 198: 4/100 refusals, KL 0.0796)	sss22213/Qwen3.8-27B-Heretic-NoRefusal
3	Japanese roleplay QLoRA on reasoning-augmented data, merged at full strength	this repository

How it was made

1. Training data: `aratako-rp-think1`

Source: Aratako/Synthetic-Japanese-Roleplay-NSFW-DeepSeek-V3-0324-20k-formatted (MIT, 19,917 Japanese roleplay conversations generated with DeepSeek-V3-0324, each with a character/scenario system message and a multi-turn dialogue).

The source has no reasoning content. To train a thinking model without erasing its thinking, the last assistant turn of every conversation was regenerated by the base model (Qwen3.8-27B-Heretic-NoRefusal, bf16, thinking mode, served through an OpenAI-compatible API):

The base model received the original system prompt and the full dialogue history up to that turn, and wrote its own <think> reasoning and its own reply. The dataset's original line for that turn was discarded, so the reasoning and the reply are always consistent and nothing in the trace refers to a "given answer". Earlier turns stay as in the dataset, which is where the roleplay style comes from.
Generation: thinking on, temperature 0.6 / top-p 0.95 / top-k 20, up to 5,120 new tokens, 16k context, seed 3407, up to 2 retries when a generation hit the length limit.
Filters: reasoning between 200 and 6,000 characters, reply at least 10 characters. 18,802 conversations passed (about 20k sampled). Passed samples average ~1,500 characters of reasoning and ~210 characters of reply.

The reasoning is stored as reasoning_content on the assistant message, which Qwen3.8's chat template renders as a real <think>…</think> block.

2. QLoRA training

Trained with LoRA Forge (an Unsloth + TRL SFT environment) on a single RTX 5090 (32 GB).

Table with columns: Setting, Value
Setting	Value
Base weights	`Qwen3.8-27B-Heretic-NoRefusal`, loaded in 4-bit (QLoRA)
LoRA	r = 32, alpha = 32, dropout 0.05
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` — language-model layers only; the vision tower is untouched
Sequence length	8,192 (no packing)

3. Merge

The adapter was merged into the bf16 Qwen3.8-27B-Heretic-NoRefusal weights with PEFT merge_and_unload at lora_alpha = 32 (the trained value, i.e. full strength; effective scale alpha / r = 1.0). Output: two safetensors shards, ~51 GB, Qwen3_5ForConditionalGeneration, unquantized.

Usage

Use it exactly like Qwen/Qwen3.8-27B — same chat template, same enable_thinking / reasoning_effort controls. Leave thinking on for roleplay; that is the mode the model was trained in. Put the character sheet and scenario in the system message, as the training data does.

python
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "sss22213/Qwen3.8-27B-Heretic-JP-Roleplay-NSFW"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto", device_map="auto")

messages = [
    {"role": "system", "content": "あなたは…（キャラクター設定と状況）"},
    {"role": "user", "content": "…"},
]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt",
).to(model.device)
output = model.generate(**inputs, max_new_tokens=2048)
print(processor.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

The full checkpoint is ~51 GB; for a single consumer GPU, quantize it (GGUF via llama.cpp / Ollama). Because the vision encoder is the stock Qwen3.8-27B one, a GGUF import can reuse the official Qwen3.8 vision projector for image input.

Limitations and caveats

Adult content. The training data is explicitly NSFW and the base model has had its refusals removed. The model will write sexual, violent and otherwise disturbing content when the roleplay calls for it, and will not decline harmful requests. You are responsible for how you use it.
Japanese roleplay is the only thing it was trained on. English and Chinese still work (inherited from the base), but the fine-tune was not evaluated on them, nor on general assistant tasks, coding or benchmarks. Expect some drift from the base on non-roleplay use.
No formal evaluation. Beyond the training loss and manual checks that reasoning length and roleplay quality held up, no benchmark numbers exist for this checkpoint. The refusal and KL numbers above belong to the base NoRefusal model, measured before this fine-tune.
Synthetic data. The source conversations were generated by DeepSeek-V3-0324 and the reasoning traces by the base model itself; the model inherits their style, tics and mistakes.
Vision was not trained or evaluated. Image input is available because the weights are intact, but nothing in the roleplay data contains images.

Acknowledgements

Qwen for Qwen3.8-27B.
Aratako for the roleplay dataset.
p-e-w/heretic (Philipp Emanuel Weidmann) for the abliteration tool.
Unsloth for the training stack.

Released under Apache-2.0, like the base model; the training data is MIT.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

sss22213

Model Tree

Base

sss22213/Qwen3.8-27B-Heretic-NoRefusal

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Lineage

Table with columns: Step, What, Where
Step	What	Where
1	Qwen3.8-27B (Apache-2.0, vision-language, thinking)	Qwen/Qwen3.8-27B
2	Refusal direction ablated with Heretic (trial 198: 4/100 refusals, KL 0.0796)	sss22213/Qwen3.8-27B-Heretic-NoRefusal
3	Japanese roleplay QLoRA on reasoning-augmented data, merged at full strength	this repository

How it was made

1. Training data: `aratako-rp-think1`

The base model received the original system prompt and the full dialogue history up to that turn, and wrote its own <think> reasoning and its own reply. The dataset's original line for that turn was discarded, so the reasoning and the reply are always consistent and nothing in the trace refers to a "given answer". Earlier turns stay as in the dataset, which is where the roleplay style comes from.
Generation: thinking on, temperature 0.6 / top-p 0.95 / top-k 20, up to 5,120 new tokens, 16k context, seed 3407, up to 2 retries when a generation hit the length limit.
Filters: reasoning between 200 and 6,000 characters, reply at least 10 characters. 18,802 conversations passed (about 20k sampled). Passed samples average ~1,500 characters of reasoning and ~210 characters of reply.

The reasoning is stored as reasoning_content on the assistant message, which Qwen3.8's chat template renders as a real <think>…</think> block.

2. QLoRA training

Trained with LoRA Forge (an Unsloth + TRL SFT environment) on a single RTX 5090 (32 GB).

Table with columns: Setting, Value
Setting	Value
Base weights	`Qwen3.8-27B-Heretic-NoRefusal`, loaded in 4-bit (QLoRA)
LoRA	r = 32, alpha = 32, dropout 0.05
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` — language-model layers only; the vision tower is untouched
Sequence length	8,192 (no packing)

3. Merge

Usage

python
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "sss22213/Qwen3.8-27B-Heretic-JP-Roleplay-NSFW"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto", device_map="auto")

messages = [
    {"role": "system", "content": "あなたは…（キャラクター設定と状況）"},
    {"role": "user", "content": "…"},
]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt",
).to(model.device)
output = model.generate(**inputs, max_new_tokens=2048)
print(processor.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Limitations and caveats

Adult content. The training data is explicitly NSFW and the base model has had its refusals removed. The model will write sexual, violent and otherwise disturbing content when the roleplay calls for it, and will not decline harmful requests. You are responsible for how you use it.
Japanese roleplay is the only thing it was trained on. English and Chinese still work (inherited from the base), but the fine-tune was not evaluated on them, nor on general assistant tasks, coding or benchmarks. Expect some drift from the base on non-roleplay use.
No formal evaluation. Beyond the training loss and manual checks that reasoning length and roleplay quality held up, no benchmark numbers exist for this checkpoint. The refusal and KL numbers above belong to the base NoRefusal model, measured before this fine-tune.
Synthetic data. The source conversations were generated by DeepSeek-V3-0324 and the reasoning traces by the base model itself; the model inherits their style, tics and mistakes.
Vision was not trained or evaluated. Image input is available because the weights are intact, but nothing in the roleplay data contains images.

Acknowledgements

Qwen for Qwen3.8-27B.
Aratako for the roleplay dataset.
p-e-w/heretic (Philipp Emanuel Weidmann) for the abliteration tool.
Unsloth for the training stack.

Released under Apache-2.0, like the base model; the training data is MIT.

Qwen3.8-27B-Heretic-JP-Roleplay-NSFW

README

Lineage

How it was made

1. Training data: aratako-rp-think1

2. QLoRA training

3. Merge

Usage

Limitations and caveats

Acknowledgements

Explore FriendliAI today

README

Lineage

How it was made

1. Training data: aratako-rp-think1

2. QLoRA training

3. Merge

Usage

Limitations and caveats

Acknowledgements

1. Training data: `aratako-rp-think1`

1. Training data: `aratako-rp-think1`