Model files
The checkpoint consists of model-00001-of-00011.safetensors through
model-00011-of-00011.safetensors, with model.safetensors.index.json, configuration,
tokenizer, processor and chat templates. The weight files total 51.61 GB in decimal units.
The original vision weights and configuration are retained; the evaluations described below cover text only.
Download and usage
hf download EternalDragon/G4-MeroMero-26B-A4B-Heretic2 --local-dir ./MeroMero-Heretic2
Load the checkpoint with a Gemma 4-capable version of Transformers.
Use the included chat_template.jinja for formatting conversations.
The original Gemma4-Think.json and Gemma4-NoThink.json profiles are also included.
Heretic modification
- Base revision:
475ecf2da8d4e758343541832b3b129b9d4f93da.
- Heretic 2.0.0, Arbitrary-Rank Ablation, rank 50, seed 42.
- Attention output projections and shared MLP down projections were eligible for ablation.
- Selected parameters: start layer 15, end layer 26, preserve-good-behavior weight 0.3274,
steer-bad-behavior weight 0.0005, overcorrect-relative weight 0.6647, neighbor count 15.
- The selected BF16 merge is the reproduced starting candidate retained after the extended search.
Optimization results
The merged BF16 model scored 9/100 on Heretic's optimization refusal set,
with first-token KL 0.014121. Those prompts were used for optimization;
these are not independent quality or generalization benchmarks.
The optimization record is in evaluation.json.
provenance.json records the parent, ablation parameters and checkpoint checksums.
SHA256SUMS covers the current released files, excluding itself.
License and attribution
Apache-2.0, matching the upstream MeroMero and Gemma 4 repository metadata.
See LICENSE and NOTICE for the license and attribution.