MN-12B-Nymphaea-RP API & Inference Endpoint

Training Notes

Trained on the latest iteration of my Darkmere dataset. This version features expanded genre variety, built upon a mix of manually curated synthetics and human-written stories.

[!IMPORTANT] The base weights are abliterated via Heretic prior to fine-tuning, so this fine-tune is quite uncensored.

Method:

Training Method: DoRA (Weight-Decomposed LoRA)
Target Modules all-linear
LoRA Rank: 64
LoRA Alpha: 64
LoRA Dropout: 0.05

Hyperparameters:

Batch Size: 2 (Per-device)
Gradient Accumulation: 2
Epochs: 2
Learning Rate: 1e-4
Optimizer: adamw_torch_fused
LR Scheduler: cosine
Noise Level: neftune_noise_alpha=5

Special Thanks

This fine-tune wouldn't be possible without the incredible work of the community:

p-e-w for developing Heretic - an essential tool for censorship removal.
SicariusSicariiStuff for developing SLOP_Detector script.
Mistral AI for their Mistral Nemo weights.
AMD for their Instinct™ MI300X GPU.

Training Notes

Trained on the latest iteration of my Darkmere dataset. This version features expanded genre variety, built upon a mix of manually curated synthetics and human-written stories.

[!IMPORTANT] The base weights are abliterated via Heretic prior to fine-tuning, so this fine-tune is quite uncensored.

Method:

Training Method: DoRA (Weight-Decomposed LoRA)
Target Modules all-linear
LoRA Rank: 64
LoRA Alpha: 64
LoRA Dropout: 0.05

Hyperparameters:

Batch Size: 2 (Per-device)
Gradient Accumulation: 2
Epochs: 2
Learning Rate: 1e-4
Optimizer: adamw_torch_fused
LR Scheduler: cosine
Noise Level: neftune_noise_alpha=5

Special Thanks

This fine-tune wouldn't be possible without the incredible work of the community:

p-e-w for developing Heretic - an essential tool for censorship removal.
SicariusSicariiStuff for developing SLOP_Detector script.
Mistral AI for their Mistral Nemo weights.
AMD for their Instinct™ MI300X GPU.

MN-12B-Nymphaea-RP

README

Training Notes

Special Thanks

Explore FriendliAI today

README

Training Notes

Special Thanks