Update

This model is more censored than Goetia 31B as tested via Q0 Benchmark.

Donors were first scanned via the della_audit script in order to gauge their influence on the merge. Weights were then modified to allow for balanced distribution of each model's influence within the MLP layers.

bat
Copy code
[DELLA Audit] Layer: model.language_model.layers.25.mlp.down_proj.weight | Lambda=1.00
  [BASE] google--gemma-4-31B-it                            
  BeaverAI--Artemis-31B-v1h-GGUF                    : ██████                                              13.8% (W:0.10 D:0.90 N:2.82 E:0.09)
  ConicCat--Gemma4-GarnetV2-31B                     : ████████                                            16.4% (W:0.50 D:0.90 N:0.67 E:0.09)
  Darkhn-Gemma-4-31B-Animus-V14.0                   : ███████                                             15.7% (W:0.50 D:0.90 N:0.64 E:0.09)
  Lambent--Fabled-Gemma4-31B                        : ██████                                              13.6% (W:0.10 D:0.90 N:2.77 E:0.09)
  LatitudeGames--Equinox-31B                        : ██████                                              13.6% (W:0.10 D:0.90 N:2.77 E:0.09)
  llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-: ██████                                              13.3% (W:0.15 D:0.90 N:1.81 E:0.09)
  virtuous7373--Gemma-4-Harmonia-31B                : ██████                                              13.6% (W:0.15 D:0.90 N:1.85 E:0.09)

Merge Details

Merge Method

This model was merged using the DELLA merge method using B:/31B/google--gemma-4-31B-it as a base.

This merge also required the sparsity v3 patch, the notes of which are here.

Models Merged

The following models were included in the merge:

• B:/31B/Darkhn-Gemma-4-31B-Animus-V14.0
• B:/31B/LatitudeGames--Equinox-31B
• B:/31B/Lambent--Fabled-Gemma4-31B
• B:/31B/virtuous7373--Gemma-4-Harmonia-31B
• B:/31B/ConicCat--Gemma4-GarnetV2-31B
• B:/31B/llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic
• B:/31B/BeaverAI--Artemis-31B-v1h-GGUF

Configuration

The following YAML configuration was used to produce this model:

yaml
Copy code
architecture: Gemma4ForConditionalGeneration
base_model: B:/31B/google--gemma-4-31B-it
models:
  - model: B:/31B/BeaverAI--Artemis-31B-v1h-GGUF
    parameters:
      weight: 0.1
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/Lambent--Fabled-Gemma4-31B
    parameters:
      weight: 0.1
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/LatitudeGames--Equinox-31B
    parameters:
      weight: 0.1
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic
    parameters:
      weight: 0.15
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/virtuous7373--Gemma-4-Harmonia-31B
    parameters:
      weight: 0.15
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/Darkhn-Gemma-4-31B-Animus-V14.0
    parameters:
      weight: 0.5
      density: 0.9
      epsilon: 0.09
  - model: B:/31B/ConicCat--Gemma4-GarnetV2-31B
    parameters:
      weight: 0.5
      density: 0.9
      epsilon: 0.09
merge_method: della
parameters:  
  lambda: 1.0
  normalize: false
  int8_mask: false
  rescale: true
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: union
chat_template: auto