Training Configuration
SFT Parameters
Table with columns: Parameter, Value| Parameter | Value |
|---|
| Epochs | 3 |
| Effective Batch Size | 4 |
| Learning Rate | 2e-4 |
| LR Scheduler | Linear |
| Warmup Steps | 10 |
| Optimizer | AdamW 8-bit |
| Weight Decay | 0.01 |
LoRA Rank (r) | 32 |
| LoRA Alpha | 64 |
Abliteration Parameters
Table with columns: Parameter, Value| Parameter | Value |
|---|
| Layers Abliterated | 100% |
| Experts Abliterated | 100% |
| Scale | 0.95 |
| Winsorization | 0.995 |
⚠️ Limitations & Responsible Use
- This model has reduced refusal behaviour on therapeutic and dark content. It is not suitable for general deployment without guardrails.
- Not a replacement for human therapeutic support.
- Patent pending (IP Australia).
user feedback indicates temperature settings between 0.9 and 1.1 particularly for AuDHD populations
Recommended System Prompt
- Incude your thresholds for crisis escalation, such as "If user states x, then y"
- Encourage the model to minimise its responses, the Base model and Gemma4 are quite verbose otherwise.
- If you require a different language consistently, ensure this is in the System Prompt as the model can forget.
- We recommend stating the "boundaries" you expect the model to operate within. For example, the tone and register, if certain material might trigger warnings or support.
Uploaded finetuned model
- Developed by: senaro
- License: apache-2.0
- Finetuned from model : senaro/atlas-trm13-gemma4-26b
This gemma4 model was trained 2x faster with Unsloth and Huggingface's TRL library.