Merge Details

Merge Method

This model was merged using the DELLA merge method using mistralai/Mistral-Nemo-Instruct-2407 as a base.

Models Merged

The following models were included in the merge:

• mistralai/Mistral-Nemo-Instruct-2407
• DarkArtsForge/MN-Raven-12B-v1
• Lambent/Arsenic-Shahrazad-12B-v4.4
• Retreatcost/Evertide-RX-12B
• WokeAI/Tankie-DPE-12B-SFT-v2

Configuration

The following YAML configuration was used to produce this model:

yaml
Copy code
architecture: MistralForCausalLM
base_model: B:\12B\mistralai--Mistral-Nemo-Instruct-2407
models:
  - model: B:\12B\MN-Raven-12B-v1
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\Lambent--Arsenic-Shahrazad-12B-v4.4
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\Retreatcost--Evertide-RX-12B
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\WokeAI--Tankie-DPE-12B-SFT-v2
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.09
merge_method: della
parameters:  
  lambda: 1.0
  normalize: false
  int8_mask: false
  rescale: true
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: union
chat_template: auto