GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
atakankutalp
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
atakankutalp
Model Tree
google/gemma-4-E2B-it
Input Modalities
Output Modalities
Supported Functionality
| Parameter | Value |
|---|---|
| direction_index | per layer |
| attn.o_proj.max_weight | 1.3787 |
| attn.o_proj.max_weight_position | 29.1532 |
| attn.o_proj.min_weight | 1.1770 |
| attn.o_proj.min_weight_distance | 14.3499 |
| mlp.down_proj.max_weight | 1.4760 |
| mlp.down_proj.max_weight_position | 21.4004 |
| mlp.down_proj.min_weight | 1.4737 |
| mlp.down_proj.min_weight_distance | 18.8598 |
| Metric | This model | Original model (google/gemma-4-E2B-it) |
|---|---|---|
| KL divergence | 0.3269 | 0 (by definition) |
| Refusals | 21/100 | 99/100 |
| Setting | Value |
| Base model | google/gemma-4-E2B-it |
| Trials | 200 (n_startup_trials 60) |
| Seed | 3407 |
| dtype / quantization | auto (bfloat16) / none |
| Row normalization | full (LoRA rank 3) |
| Orthogonalize direction | yes |
| Batch size | 16 (auto-selected, --max-batch-size 16) |
| Selected trial | Pareto front trial with lowest refusals (21/100, KL 0.3269) |
[!NOTE] The randomized SVD (
torch.svd_lowrank) inside Heretic'sfullrow-normalization path was executed on the CPU rather than the MPS backend. The PyTorch MPS build ofsvd_lowranktakes minutes per call instead of milliseconds, which would have made each optimization trial take hours. The RNG is reseeded (torch.manual_seed(3407)) immediately before every call, so results remain deterministic and reproducible.
[!WARNING] These weight files have been modified.
This repository contains
google/gemma-4-E2B-itafter directional ablation ("abliteration"), performed automatically with Heretic v1.4.0. A refusal direction was identified in the model's residual stream and projected out of every attention out-projection (o_proj) and MLP down-projection (down_proj) across all transformer layers, with norm-preserving (full) row normalization, then merged into the base weights.The model's safety alignment has been deliberately reduced. Refusals on sensitive prompts are substantially lower than the original checkpoint, and outputs are not safety-filtered. Use accordingly.
| Original | This repository | |
|---|---|---|
| Base model | google/gemma-4-E2B-it | same weights, ablated |
| Refusals on harmful prompts | 99/100 | 21/100 |
| Safety alignment | intact | deliberately reduced |
| License | Apache 2.0 | same |
python
from transformers import AutoModelForImageTextToText, AutoProcessorimport torch model_id = "atakankutalp/gemma-4-E2B-it-heretic"processor = AutoProcessor.from_pretrained(model_id)model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": "Who are you?"}]inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=128)print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))mlabonne/harmless_alpaca (train[:400], evaluated on test[:100])mlabonne/harmful_behaviors (train[:400], evaluated on test[:100])Quantized GGUF builds of this exact ablated checkpoint, converted with
llama.cpp's convert_hf_to_gguf.py and quantized
with llama-quantize. Useful for Windows / CPU inference (Ollama, LM Studio, llama.cpp, koboldcpp, ...).
| File | Quant | Size |
|---|---|---|
gguf/gemma4-e2b-it-heretic.f16.gguf | F16 | 9.27 GB |
gguf/gemma4-e2b-it-heretic.Q8_0.gguf | Q8_0 | 4.95 GB |
gguf/gemma4-e2b-it-heretic.Q6_K.gguf | Q6_K | 3.83 GB |
gguf/gemma4-e2b-it-heretic.Q5_K_M.gguf | Q5_K_M | 3.62 GB |
bash
llama-cli -hf atakankutalp/gemma-4-E2B-it-heretic --conversation \ --temp 0.1 --top-k 50 --repeat-penalty 1.1[!NOTE] The GGUF files are text-only: the vision tower of
Gemma4ForConditionalGenerationis not included. Use the safetensors files in this repository for multimodal use.
gguf/gemma4-e2b-it-heretic.Q4_K_M.gguf |
| Q4_K_M |
| 3.42 GB |