What this is
The body is composed with an operator I am calling an isometric merge. Each source delta is reduced to the rotation it induces on the base weight inside the subspace it actually moved; those rotations are averaged on the orthogonal group and applied as W → W + Z(R − I)Zᵀ W. Because R is orthogonal, the merged weight carries the same norm it started with, to machine precision. There is nothing to restore afterward.
The conventional path interpolates the deltas and rescales the result back to a target norm. That is not an isometry, and the leftover magnitude has to be put back by hand. In my runs that final rescale is exactly what produces repetition loops — the same failure that appears when you edit late-layer behavior with a crude magnitude change instead of a norm-preserving one. The architecture is fairly clear about which edit it prefers; the isometric merge takes it at its word, and stays coherent under load where additive and slerp merges of the same ingredients did not.
Two further choices follow from the same view. The subspace each rotation acts in is chosen in the activation metric Δ · H^½, with H = E[xxᵀ] harvested from the base — the directions the activations actually use, not the raw Frobenius directions — and each source is weighted by how much it moves the states the model visits rather than by the size of its weight change, so that a large but mostly-noisy dense tune does not drown the low-norm adapters that carry the actual flavor. Pure rotation by itself is nearly inert, so a bounded radial term reintroduces finetune strength, placed in the low-rank signal head rather than the quantization-noise tail, with an on-policy generation check as the ceiling on how far it is allowed to go.
What is in it
Rotated into the body:
Table with columns: source, contribution| source | contribution |
|---|
LatitudeGames/Equinox-31B | full-parameter SFT, the broad body |
EganAI/gemma-4-31B-opus-Reasoning-Distilled | reasoning-distilled full SFT |
zerofata/G4-MeroMero-31B | roleplay |
AuriAetherwiing/G4-31B-Musica-v1-lora | roleplay |
nbeerbower/Gemma4-Gutenberg-31B-LoRA | literary prose |
Lambent/Fabled-Gemma4-31B-Adapter | dark and creative continuation |
ConicCat/Gemma4-Garnet-31B | reasoning and creative |
54rt1n/heretic-94 | sparse o_proj abliteration |
Routed in directly, not merged: input embeddings from
google/gemma-4-31B-it-qat-q4_0-unquantized, and lm_head from
Gryphe/Gemma-4-31B-StyleTune (the model is untied).
Quantization
This model should compress well and run well at low precision, and the reasons are structural: The input embeddings come from Google's quantization-aware-trained Gemma-4 checkpoint; they were trained to survive q4_0 rounding, so taking them back down to four bits costs far less than quantizing ordinary weights. And because the merge is norm-preserving, it introduces no inflated weight magnitudes for a low-bit quantizer to clip or to spend its dynamic range on. This is the same property that makes structured hidden state survive rotation and aggressive quantization in the first place: what matters here is direction, and direction is what these formats keep.
What to expect
The merged model's first-token distribution moves toward the sources it was built from, and does so selectively — it leans into the directions where a source has signal and stays near the base where it does not. Over long generations it holds: on four-thousand-token rollouts it kept its footing on dark fantasy, on literary prose, and on a multi-step reasoning problem that it solved correctly, in a setting where the louder variant of the same merge lost coherence partway through.
It is a roleplay and creative-writing model first. It will reason and it will code, but that is not what it was tuned for.
Caveats
This is an experimental merge built around the isometric merge algorithm described above. The radial energy knob still matters, and too much of it can bring back the same instability the rotation was meant to avoid.
The model is also untied: the input embeddings and lm_head are routed in from different sources. Preserve that layout when converting or quantizing.
Usage
Standard Gemma instruct chat template, included. It prefers a little room: temperature around 0.8–1.0, top-p around 0.95. Untied head, bf16, 17 shards.
If you have repetition problems, dialing up min-p helps. Gryphe's StyleTune head causes quite a bit of instability for the sampler. It also helps to use instruction + thinking.
Citation
@misc{maldv2026isometryrp, title = {Gemma 4 31B Isometry RP}, author = {Praxis Maldevide}, year = {2026}, note = {Isometric, norm-preserving, activation-weighted merge of google/gemma-4-31B-it}, howpublished = {\url{https://huggingface.co/maldv/Gemma-4-31B-Isometry-RP}}}
