activation-oracle-legacy-gemma-4-E2B-it

Why This Checkpoint Is Legacy

This model was trained before the current Gemma 4 AO injection convention was adopted. In practice, that means:

• it uses an older activation transport / injection recipe
• it uses an older layer-selection convention
• it should be treated as a historical artifact rather than the default Gemma 4 AO

Classification-style evaluations may still look reasonable, but that does not make this checkpoint the right choice for current Gemma 4 AO work.

On this small dense Gemma 4 model, the mismatch is larger because the architecture includes per-layer side inputs and shared-KV structure that are not well captured by the older residual-only transport recipe.

When To Use It

Use this checkpoint only if you specifically want to:

• reproduce earlier Gemma 4 AO results
• compare older and newer AO training conventions
• inspect how the legacy recipe behaves

For new Gemma 4 AO experiments, use a checkpoint trained with the current standard instead.

Quick Start

python
Copy code
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-4-E2B-it",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-E2B-it")

model = PeftModel.from_pretrained(base_model, "EvilScript/activation-oracle-legacy-gemma-4-E2B-it")
model.eval()

Legacy Training Details

Related Resources

• Paper: Confidence and Calibration of Activation Oracles (arXiv:2605.26045)
• Code: activation_oracles