Abliteration parameters

Performance

---

🐊 Agares 31B v1

Agares is a quick merge test for the upcoming Goetia 31B. The idea was to see if Artemis v1h would destroy the merge or not.

While significantly degraded on its own when using Q8_0 source, upon being merged, most of the damage appears to be mitigated by the merge method used, in this case Della's selective Magnitude Pruning.

Censorship levels weren't tested, though in theory it should have somewhat less refusals due to normalize: false and having at least one heretic donor.

Update

This model is more censored than Goetia 31B as tested via Q0 Benchmark.

Donors were first scanned via the della_audit script in order to gauge their influence on the merge. Weights were then modified to allow for balanced distribution of each model's influence within the MLP layers.

bat
Copy code
[DELLA Audit] Layer: model.language_model.layers.25.mlp.down_proj.weight | Lambda=1.00
  [BASE] google--gemma-4-31B-it                            
  BeaverAI--Artemis-31B-v1h-GGUF                    : ██████                                              13.8% (W:0.10 D:0.90 N:2.82 E:0.09)
  ConicCat--Gemma4-GarnetV2-31B                     : ████████                                            16.4% (W:0.50 D:0.90 N:0.67 E:0.09)
  Darkhn-Gemma-4-31B-Animus-V14.0                   : ███████                                             15.7% (W:0.50 D:0.90 N:0.64 E:0.09)
  Lambent--Fabled-Gemma4-31B                        : ██████                                              13.6% (W:0.10 D:0.90 N:2.77 E:0.09)
  LatitudeGames--Equinox-31B                        : ██████                                              13.6% (W:0.10 D:0.90 N:2.77 E:0.09)
  llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-: ██████                                              13.3% (W:0.15 D:0.90 N:1.81 E:0.09)
  virtuous7373--Gemma-4-Harmonia-31B                : ██████                                              13.6% (W:0.15 D:0.90 N:1.85 E:0.09)

Merge Details

Merge Method

This model was merged using the DELLA merge method using B:/31B/google--gemma-4-31B-it as a base.

This merge also required the sparsity v3 patch, the notes of which are here.

Models Merged

The following models were included in the merge:

• B:/31B/Darkhn-Gemma-4-31B-Animus-V14.0
• B:/31B/LatitudeGames--Equinox-31B
• B:/31B/Lambent--Fabled-Gemma4-31B
• B:/31B/virtuous7373--Gemma-4-Harmonia-31B
• B:/31B/ConicCat--Gemma4-GarnetV2-31B
• B:/31B/llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic
• B:/31B/BeaverAI--Artemis-31B-v1h-GGUF

Configuration

The following YAML configuration was used to produce this model:

yaml
Copy code
architecture: Gemma4ForConditionalGeneration
base_model: B:/31B/google--gemma-4-31B-it
models:
  - model: B:/31B/BeaverAI--Artemis-31B-v1h-GGUF
    parameters:
      weight: 0.1
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/Lambent--Fabled-Gemma4-31B
    parameters:
      weight: 0.1
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/LatitudeGames--Equinox-31B
    parameters:
      weight: 0.1
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic
    parameters:
      weight: 0.15
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/virtuous7373--Gemma-4-Harmonia-31B
    parameters:
      weight: 0.15
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/Darkhn-Gemma-4-31B-Animus-V14.0
    parameters:
      weight: 0.5
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/ConicCat--Gemma4-GarnetV2-31B
    parameters:
      weight: 0.5
      density: 0.9
      epsilon: 0.09
merge_method: della
parameters:  
  lambda: 1.0
  normalize: false
  int8_mask: false
  rescale: true
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: union
chat_template: auto