Naphula

Goetia-26B-A4B-v1.6

Deploy Dedicated

README

License: apache-2.0

Merge Details

Merge Method

This model was merged using the moe_della merge method using google/gemma-4-26B-A4B-it as a base.

Models Merged

The following models were included in the merge:

⚙️ Configuration

yaml
architecture: Gemma4ForConditionalGeneration
base_model: B:\26B\google--gemma-4-26B-A4B-it # instruct
models:
  - model: B:\26B\ApocalypseParty_G4-26B-SFT-6
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\AuriAetherwiing_G4-26B-A4B-Musica-v1
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ayagaya--gem4_26B_adapter-safetensors
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.0
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\BeaverAI--Orion-26B-A4B-v1g-Q8_0
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Calplus--GemmaWiki-Gemma-4-26b-a4b-safetensors
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.0
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Darkhn_Gemma-4-26B-A4B-Animus-V14.1-FFT
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\el4--Xenon-26B-A4B-safetensors
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\electroglyph--gemma4-26b-fiction-bf16
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Gryphe--Pantheon-Reasoning-26B-A4B-1.1-V2
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\haster--Gemma-4-26BA4B-Sonnet-3.7-Distilled-v2.Q8_0
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\hotdogs--gemma4-26b-claude-opus-reasoning-safetensors
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Jackrong--Gemopus-4-26B-A4B-it
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\khazic--gemma4-26b-a4b-sft-v5
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Locutusque--Esmeralda-Gemma4-26B-A4B
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\nbeerbower--Gemma4-Gutenberg-26B-A4B-safetensors
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--Dark-Scarlett-v1.0-26B-A4B
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--For-Her-Darkside-26B-A4B-v1.4
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--Heimdallr-26B-A4B-v0.35-Q8_0
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--Melody1437-26B-A4B-v2.0
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--Omega-Evolution-26B-A4B-v3.0-Q8_0
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--Serenity-26B-A4B
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\SubMaroon--Boulesis-26B-A4B
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.0
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\TeichAI--gemma-4-26B-A4B-it-Claude-Opus-Distill-v2
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.0 # the smallest non-zero mlp distance "0.1048" gets nulled to keep mlp donor count at 20
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.0
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\TheDrummer--Orion-26B-A4B-v1 # Use this chat_template.jinja
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.5
        - filter: "attn"
          value: 0.0
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.5
        - value: 0.00
      density: 1.0
      epsilon: 0.0
  - model: B:\26B\UncannyEcho--Aura-Gaiden-Medium-v1.1-BF16
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.0
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\zerofata--G4-MeroMero-26B-A4B
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Gryphe--Gemma-4-26B-A4B-StyleTune-V2
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.5
        - filter: "attn"
          value: 0.0
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.5
        - value: 0.0
      density: 1.0
      epsilon: 0.0
merge_method: moe_della
parameters:  
  lambda: 1.0
  # normalize: true # deprecated
  normalize_weights: false
  normalize_router: true
  int8_mask: false
  rescale: true
  router_strategy: della # average # random_init
  blend_experts: true
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: union
chat_template: auto
name: 📜 Goetia 26B A4B v1.6A

yaml
architecture: Gemma4ForConditionalGeneration
base_model: B:\26B\Goetia-26B-A4B-v1.6A # instruct
models:
  - model: B:\26B\TheDrummer--Orion-26B-A4B-v1 # Use this chat_template.jinja
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.5
        - filter: "attn"
          value: 0.0
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.5
        - value: 0.0
      density: 1.0
      epsilon: 0.0
  - model: B:\26B\Gryphe--Pantheon-Reasoning-26B-A4B-1.1-V2
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.5
        - filter: "attn"
          value: 0.0
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.5
        - value: 0.0
      density: 1.0
      epsilon: 0.0
merge_method: moe_della # https://github.com/EldritchLabs/mergekit-exp
parameters:  
  lambda: 1.0
  # normalize: true # deprecated
  normalize_weights: false
  normalize_router: true
  int8_mask: false
  rescale: true
  router_strategy: della # average # random_init
  blend_experts: true
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: union
chat_template: auto
name: 📜 Goetia 26B A4B v1.6

This model is not currently uncensored. There are refusals and standard jailbreak resistance.

Advanced Jailbreaks can be used to bypass censorship. It can also be decensored using Heretic, although I have not had the time or funds to uncensor it yet.

Special Thanks

Thanks to @SubMaroon, @Vortex5, @redaihf, @Phil2Sat and others who helped with debugging and testing.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Naphula

Model Tree

Base

AuriAetherwiing/G4-26B-A4B-Musica-v1

Base

ReadyArt/Serenity-26B-A4B

Base

electroglyph/gemma4-26b-fiction-bf16

Base

Jackrong/Gemopus-4-26B-A4B-it

Base

TeichAI/gemma-4-26B-A4B-it-Claude-Opus-Distill-v2

Base

Darkhn/Gemma-4-26B-A4B-Animus-V14.1-FFT

Base

TheDrummer/Orion-26B-A4B-v1

Base

SubMaroon/Boulesis-26B-A4B

Base

ReadyArt/Dark-Scarlett-v1.0-26B-A4B

Base

nbeerbower/Gemma4-Gutenberg-26B-A4B-lora

Base

ApocalypseParty/G4-26B-SFT-6

Base

Gryphe/Pantheon-Reasoning-26B-A4B-1.1-V2

Base

ReadyArt/For-Her-Darkside-26B-A4B-v1.4

Base

UncannyEcho/Aura-Gaiden-Medium-v1.1-BF16

Base

google/gemma-4-26B-A4B-it

Base

Locutusque/Esmeralda-Gemma4-26B-A4B

Base

BeaverAI/Orion-26B-A4B-v1g-GGUF

Base

Calplus/GemmaWiki-Gemma-4-26b-a4b

Base

ReadyArt/Melody1437-26B-A4B-v2.0

Base

Gryphe/Gemma-4-26B-A4B-StyleTune-V2

Base

zerofata/G4-MeroMero-26B-A4B

Base

el4/Xenon-26B-A4B

Base

ReadyArt/Heimdallr-26B-A4B-v0.35-GGUF

Base

haster/Gemma-4-26BA4B-Sonnet-3.7-Distilled-v2

Base

ReadyArt/Omega-Evolution-26B-A4B-v3.0-GGUF

Base

khazic/gemma4-26b-a4b-sft-v5

Merged

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Merge Details

Merge Method

This model was merged using the moe_della merge method using google/gemma-4-26B-A4B-it as a base.

Models Merged

The following models were included in the merge:

⚙️ Configuration

yaml
architecture: Gemma4ForConditionalGeneration
base_model: B:\26B\google--gemma-4-26B-A4B-it # instruct
models:
  - model: B:\26B\ApocalypseParty_G4-26B-SFT-6
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\AuriAetherwiing_G4-26B-A4B-Musica-v1
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ayagaya--gem4_26B_adapter-safetensors
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.0
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\BeaverAI--Orion-26B-A4B-v1g-Q8_0
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Calplus--GemmaWiki-Gemma-4-26b-a4b-safetensors
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.0
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Darkhn_Gemma-4-26B-A4B-Animus-V14.1-FFT
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\el4--Xenon-26B-A4B-safetensors
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\electroglyph--gemma4-26b-fiction-bf16
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Gryphe--Pantheon-Reasoning-26B-A4B-1.1-V2
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\haster--Gemma-4-26BA4B-Sonnet-3.7-Distilled-v2.Q8_0
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\hotdogs--gemma4-26b-claude-opus-reasoning-safetensors
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Jackrong--Gemopus-4-26B-A4B-it
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\khazic--gemma4-26b-a4b-sft-v5
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Locutusque--Esmeralda-Gemma4-26B-A4B
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\nbeerbower--Gemma4-Gutenberg-26B-A4B-safetensors
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--Dark-Scarlett-v1.0-26B-A4B
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--For-Her-Darkside-26B-A4B-v1.4
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--Heimdallr-26B-A4B-v0.35-Q8_0
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--Melody1437-26B-A4B-v2.0
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--Omega-Evolution-26B-A4B-v3.0-Q8_0
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\ReadyArt--Serenity-26B-A4B
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\SubMaroon--Boulesis-26B-A4B
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.0
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\TeichAI--gemma-4-26B-A4B-it-Claude-Opus-Distill-v2
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.0 # the smallest non-zero mlp distance "0.1048" gets nulled to keep mlp donor count at 20
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.0
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\TheDrummer--Orion-26B-A4B-v1 # Use this chat_template.jinja
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.5
        - filter: "attn"
          value: 0.0
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.5
        - value: 0.00
      density: 1.0
      epsilon: 0.0
  - model: B:\26B\UncannyEcho--Aura-Gaiden-Medium-v1.1-BF16
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.0
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\zerofata--G4-MeroMero-26B-A4B
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.0
        - filter: "attn"
          value: 0.04
        - filter: "mlp"
          value: 0.05
        - filter: "embed_tokens"
          value: 0.0
        - value: 0.05
      density: 0.9
      epsilon: 0.09
  - model: B:\26B\Gryphe--Gemma-4-26B-A4B-StyleTune-V2
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.5
        - filter: "attn"
          value: 0.0
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.5
        - value: 0.0
      density: 1.0
      epsilon: 0.0
merge_method: moe_della
parameters:  
  lambda: 1.0
  # normalize: true # deprecated
  normalize_weights: false
  normalize_router: true
  int8_mask: false
  rescale: true
  router_strategy: della # average # random_init
  blend_experts: true
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: union
chat_template: auto
name: 📜 Goetia 26B A4B v1.6A

yaml
architecture: Gemma4ForConditionalGeneration
base_model: B:\26B\Goetia-26B-A4B-v1.6A # instruct
models:
  - model: B:\26B\TheDrummer--Orion-26B-A4B-v1 # Use this chat_template.jinja
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.5
        - filter: "attn"
          value: 0.0
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.5
        - value: 0.0
      density: 1.0
      epsilon: 0.0
  - model: B:\26B\Gryphe--Pantheon-Reasoning-26B-A4B-1.1-V2
    parameters:
      weight:
        - filter: "lm_head"
          value: 0.5
        - filter: "attn"
          value: 0.0
        - filter: "mlp"
          value: 0.0
        - filter: "embed_tokens"
          value: 0.5
        - value: 0.0
      density: 1.0
      epsilon: 0.0
merge_method: moe_della # https://github.com/EldritchLabs/mergekit-exp
parameters:  
  lambda: 1.0
  # normalize: true # deprecated
  normalize_weights: false
  normalize_router: true
  int8_mask: false
  rescale: true
  router_strategy: della # average # random_init
  blend_experts: true
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: union
chat_template: auto
name: 📜 Goetia 26B A4B v1.6

This model is not currently uncensored. There are refusals and standard jailbreak resistance.

Advanced Jailbreaks can be used to bypass censorship. It can also be decensored using Heretic, although I have not had the time or funds to uncensor it yet.

Special Thanks

Thanks to @SubMaroon, @Vortex5, @redaihf, @Phil2Sat and others who helped with debugging and testing.

Goetia-26B-A4B-v1.6

README

Merge Details

Merge Method

Models Merged

⚙️ Configuration

This model is not currently uncensored. There are refusals and standard jailbreak resistance.

Special Thanks

Explore FriendliAI today

README

Merge Details

Merge Method

Models Merged

⚙️ Configuration

This model is not currently uncensored. There are refusals and standard jailbreak resistance.

Special Thanks