Update
This model is more censored than Goetia 31B as tested via Q0 Benchmark.
Donors were first scanned via the della_audit script in order to gauge their influence on the merge. Weights were then modified to allow for balanced distribution of each model's influence within the MLP layers.
[DELLA Audit] Layer: model.language_model.layers.25.mlp.down_proj.weight | Lambda=1.00
[BASE] google--gemma-4-31B-it
BeaverAI--Artemis-31B-v1h-GGUF : ██████ 13.8% (W:0.10 D:0.90 N:2.82 E:0.09)
ConicCat--Gemma4-GarnetV2-31B : ████████ 16.4% (W:0.50 D:0.90 N:0.67 E:0.09)
Darkhn-Gemma-4-31B-Animus-V14.0 : ███████ 15.7% (W:0.50 D:0.90 N:0.64 E:0.09)
Lambent--Fabled-Gemma4-31B : ██████ 13.6% (W:0.10 D:0.90 N:2.77 E:0.09)
LatitudeGames--Equinox-31B : ██████ 13.6% (W:0.10 D:0.90 N:2.77 E:0.09)
llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-: ██████ 13.3% (W:0.15 D:0.90 N:1.81 E:0.09)
virtuous7373--Gemma-4-Harmonia-31B : ██████ 13.6% (W:0.15 D:0.90 N:1.85 E:0.09)
Merge Details
Merge Method
This model was merged using the DELLA merge method using B:/31B/google--gemma-4-31B-it as a base.
This merge also required the sparsity v3 patch, the notes of which are here.
Models Merged
The following models were included in the merge:
- B:/31B/Darkhn-Gemma-4-31B-Animus-V14.0
- B:/31B/LatitudeGames--Equinox-31B
- B:/31B/Lambent--Fabled-Gemma4-31B
- B:/31B/virtuous7373--Gemma-4-Harmonia-31B
- B:/31B/ConicCat--Gemma4-GarnetV2-31B
- B:/31B/llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic
- B:/31B/BeaverAI--Artemis-31B-v1h-GGUF
Configuration
The following YAML configuration was used to produce this model:
architecture: Gemma4ForConditionalGeneration
base_model: B:/31B/google--gemma-4-31B-it
models:
- model: B:/31B/BeaverAI--Artemis-31B-v1h-GGUF
parameters:
weight: 0.1
density: 0.9
epsilon: 0.09
- model: B:/31B/Lambent--Fabled-Gemma4-31B
parameters:
weight: 0.1
density: 0.9
epsilon: 0.09
- model: B:/31B/LatitudeGames--Equinox-31B
parameters:
weight: 0.1
density: 0.9
epsilon: 0.09
- model: B:/31B/llmfan46--gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic
parameters:
weight: 0.15
density: 0.9
epsilon: 0.09
- model: B:/31B/virtuous7373--Gemma-4-Harmonia-31B
parameters:
weight: 0.15
density: 0.9
epsilon: 0.09
- model: B:/31B/Darkhn-Gemma-4-31B-Animus-V14.0
parameters:
weight: 0.5
density: 0.9
epsilon: 0.09
- model: B:/31B/ConicCat--Gemma4-GarnetV2-31B
parameters:
weight: 0.5
density: 0.9
epsilon: 0.09
merge_method: della
parameters:
lambda: 1.0
normalize: false
int8_mask: false
rescale: true
dtype: float32
out_dtype: bfloat16
tokenizer:
source: union
chat_template: auto