Abliteration parameters
Table with columns: Parameter, Value| Parameter | Value |
|---|
| direction_index | 41.65 |
| attn.o_proj.max_weight | 1.03 |
| attn.o_proj.max_weight_position | 39.06 |
| attn.o_proj.min_weight | 0.02 |
| attn.o_proj.min_weight_distance | 20.13 |
| mlp.down_proj.max_weight | 1.14 |
| mlp.down_proj.max_weight_position | 47.70 |
| mlp.down_proj.min_weight | 0.81 |
| mlp.down_proj.min_weight_distance | 24.55 |
Table with columns: Metric, This model, Original model (DarkArtsForge/Agares-31B-v1)| Metric | This model | Original model (DarkArtsForge/Agares-31B-v1) |
|---|
| KL divergence | 0.0085 | 0 (by definition) |
| Refusals | 17/100 | 90/100 |
🐊 Agares 31B v1
Agares is a quick merge test for the upcoming Goetia 31B. The idea was to see if Artemis v1h would destroy the merge or not.
While significantly degraded on its own when using Q8_0 source, upon being merged, most of the damage appears to be mitigated by the merge method used, in this case Della's selective Magnitude Pruning.
Censorship levels weren't tested, though in theory it should have somewhat less refusals due to normalize: false and having at least one heretic donor.
Update
This model is more censored than Goetia 31B as tested via Q0 Benchmark.
Donors were first scanned via the della_audit script in order to gauge their influence on the merge. Weights were then modified to allow for balanced distribution of each model's influence within the MLP layers.
[DELLA Audit] Layer: model.language_model.layers.25.mlp.down_proj.weight | Lambda=1.00
[BASE] google--gemma-4-31B-it
BeaverAI--Artemis-31B-v1h-GGUF : ██████ 13.8% (W:0.10 D:0.90 N:2.82 E:0.09)
ConicCat--Gemma4-GarnetV2-31B : ████████ 16.4% (W:0.50 D:0.90 N:0.67 E:0.09)
Darkhn-Gemma-4-31B-Animus-V14.0 : ███████ 15.7% (W:0.50 D:0.90 N:0.64 E:0.09)
Lambent--Fabled-Gemma4-31B : ██████ 13.6% (W:0.10 D:0.90 N:2.77 E:0.09)
LatitudeGames--Equinox-31B : ██████ 13.6% (W:0.10 D:0.90 N:2.77 E:0.09)
llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-: ██████ 13.3% (W:0.15 D:0.90 N:1.81 E:0.09)
virtuous7373--Gemma-4-Harmonia-31B : ██████ 13.6% (W:0.15 D:0.90 N:1.85 E:0.09)
Merge Details
Merge Method
This model was merged using the DELLA merge method using B:/31B/google--gemma-4-31B-it as a base.
This merge also required the sparsity v3 patch, the notes of which are here.
Models Merged
The following models were included in the merge:
- B:/31B/Darkhn-Gemma-4-31B-Animus-V14.0
- B:/31B/LatitudeGames--Equinox-31B
- B:/31B/Lambent--Fabled-Gemma4-31B
- B:/31B/virtuous7373--Gemma-4-Harmonia-31B
- B:/31B/ConicCat--Gemma4-GarnetV2-31B
- B:/31B/llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic
- B:/31B/BeaverAI--Artemis-31B-v1h-GGUF
Configuration
The following YAML configuration was used to produce this model:
architecture: Gemma4ForConditionalGeneration
base_model: B:/31B/google--gemma-4-31B-it
models:
- model: B:/31B/BeaverAI--Artemis-31B-v1h-GGUF
parameters:
weight: 0.1
density: 0.9
epsilon: 0.09
- model: B:/31B/Lambent--Fabled-Gemma4-31B
parameters:
weight: 0.1
density: 0.9
epsilon: 0.09
- model: B:/31B/LatitudeGames--Equinox-31B
parameters:
weight: 0.1
density: 0.9
epsilon: 0.09
- model: B:/31B/llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic
parameters:
weight: 0.15
density: 0.9
epsilon: 0.09
- model: B:/31B/virtuous7373--Gemma-4-Harmonia-31B
parameters:
weight: 0.15
density: 0.9
epsilon: 0.09
- model: B:/31B/Darkhn-Gemma-4-31B-Animus-V14.0
parameters:
weight: 0.5
density: 0.9
epsilon: 0.09
- model: B:/31B/ConicCat--Gemma4-GarnetV2-31B
parameters:
weight: 0.5
density: 0.9
epsilon: 0.09
merge_method: della
parameters:
lambda: 1.0
normalize: false
int8_mask: false
rescale: true
dtype: float32
out_dtype: bfloat16
tokenizer:
source: union
chat_template: auto