Merge Details
Merge Method
This model was merged using the Adaptive Unified Riemannian Annealing (AURA) merge method using Retreatcost/Mistral-Nemo-Base-2407-ChatML as a base.
Models Merged
The following models were included in the merge:
- DreadPoor/Famino-12B-Model_Stock
- EldritchLabs/MN-Starlight-Sylph-12B
- OccultAI/MN-Nazgul-12B-v1
- Retreatcost/Mistral-Nemo-Base-2407-ChatML
- shrugging-shoulders/Amberlight-Lux-12B
- WokeAI/Tankie-DPE-12B-SFT-v2
Configuration
The following YAML configuration was used to produce this model:
architecture: MistralForCausalLMbase_model: B:\12B\Retreatcost--Mistral-Nemo-Base-2407-ChatMLmodels: - model: B:\12B\DreadPoor--Famino-12B-Model_Stock parameters: weight: 0.4 density: 0.90 epsilon: 0.09 pinocchio: false - model: B:\12B\EldritchLabs--MN-Starlight-Sylph-12B parameters: weight: 0.4 density: 0.90 epsilon: 0.09 pinocchio: false - model: B:\12B\shrugging-shoulders--Amberlight-Lux-12B parameters: weight: 0.4 density: 0.90 epsilon: 0.09 pinocchio: false - model: B:\12B\WokeAI--Tankie-DPE-12B-SFT-v2 parameters: weight: 0.4 density: 0.90 epsilon: 0.09 pinocchio: false - model: B:\12B\OccultAI--MN-Nazgul-12B-v1 parameters: weight: 0.4 density: 0.90 epsilon: 0.09 pinocchio: truemerge_method: aura # v6parameters: tol: 1.0e-11 max_iter: 25 eta: 1.25 theta: 0.93 optimization_mode: "hybrid" phi: 2.0 alpha: 0.25 kappa: 0.75 jl_projs: 8 halley_strength: 1.61803 halley_period: 8 scalar_gravity: -0.025 mu: 0.5 aura_shift: true omega: 0.08 momentum: 0.25 patience: 64 sharp: 0.5 voice: 0.25 density: 0.9 epsilon: 0.09 rescale: true int8_mask: false normalize_weights: false normalize_router: true router_strategy: "della" blend_experts: true auto_pad: true skip_on_mismatch: true verbose_logging: true resume_path: "B:/mergekit-new/Aura_Cache" enable_qliphoth: true pinocchio_lambda: 0.75 inverse_pinocchio: false dynamic_pareto_pinocchio: true qliphoth_gamma: 0.5 qliphoth_lambda_enslaved: 0.5 qliphoth_svd_rank: 0 qliphoth_preserve_ratio: 1.0 save_layer_heatmaps: false audit_image_dir: "B:/mergekit-new/Aura_Heatmaps" gap_jumper: 1.0 enable_qhe: true qhe_levels: 16 qhe_intensity: 0.3 qhe_crossover: 0.15 coherence_override: 1.0 conflict_scale_override: 1.0 invert_coherence_to_chaos: truedtype: float32out_dtype: bfloat16tokenizer: source: unionchat_template: auto