Training mixture
- 50,000 training records total
- 37,500 LusyChat records
- 12,500 LESS-selected generic SFT records
- Last assistant target only is supervised; context and earlier turns are masked
- Maximum sequence length: 8,192 tokens
The 75:25 ratio is by sample count.
Adapter configuration
- LoRA rank: 32
- LoRA alpha: 64
- BF16 training
- Cosine learning-rate schedule
- Learning rate: 1e-4
- Two epochs
- Final completed checkpoint: 6,250 steps
Usage
from transformers import AutoProcessor, Gemma4UnifiedForConditionalGenerationfrom peft import PeftModel base_id = "google/gemma-4-12B-it"adapter_id = "ChatoyantAI/Gemma-4-12B-Roleplay-7525-SFT"processor = AutoProcessor.from_pretrained(adapter_id)base = Gemma4UnifiedForConditionalGeneration.from_pretrained( base_id, dtype="auto", device_map="auto")model = PeftModel.from_pretrained(base, adapter_id)
This repository contains the LoRA adapter, tokenizer and Gemma chat template. It is not a standalone merged base model. Use a Transformers/PEFT version with Gemma 4 Unified support and keep the included template.
Training data, optimizer states and private logs are not included.