GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
saidutta69
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
saidutta69
Model Tree
AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5
Input Modalities
Output Modalities
Supported Functionality
Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the Heretic repo and the original abliteration writeup for the mechanism.
Full GGUF ladder included — pick the quant that fits your card:
| Your GPU | Recommended quant | Weights |
|---|---|---|
| RTX 3060 / 4070 / 5070 (12 GB) | Q8_0 | 3.62 GB |
| RTX 4060 / 3070 (8 GB) | Q6_K | 2.80 GB |
| GTX 1660 Super / 2060 / 3050 laptop (6 GB) | Q5_K_M | 2.44 GB |
| CPU-only / Apple Silicon | Q4_K_M | 2.10 GB, fits in system RAM |
Weights only, at this model's ~3.4B native size; add ~1 GB for context. OOM? Drop one quant level. Headroom to spare? Go one up.
| Parameter | Value |
|---|---|
| direction_index | 27.16 |
| attn.o_proj.max_weight | 1.49 |
| attn.o_proj.max_weight_position | 38.22 |
| attn.o_proj.min_weight | 1.42 |
| attn.o_proj.min_weight_distance | 23.26 |
| mlp.down_proj.max_weight | 1.47 |
| mlp.down_proj.max_weight_position | 25.37 |
| mlp.down_proj.min_weight | 0.61 |
| Metric | This model | Original model (AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5) |
|---|---|---|
| KL divergence | 0.0111 | 0 (by definition) |
| Refusals | 5/100 | 96/100 |
Refusals drop from 96 to 5 out of 100 adversarial prompts at a KL cost of 0.0111 — the Parable reasoning gains (agent artifacts 0/34, Fable 5 traces) are preserved.
Made with ❤️ by RACER IS OP — follow for more uncensored models
| File | Format | Size |
|---|---|---|
model-00001-of-00002.safetensors + model-00002-of-00002.safetensors | BF16 Safetensors (sharded) | ~6.8 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-F16.gguf | GGUF F16 | 6.81 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q2_K.gguf | GGUF Q2_K | 1.37 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-IQ3_S.gguf | GGUF IQ3_S |
Full 15-file GGUF ladder (F16 + 14 quants) produced with llama.cpp (GraniteForCausalLM architecture). The text-only GGUFs load in llama.cpp / Ollama / LM Studio / Jan directly. For transformers/vLLM use the Safetensors above. Run llama serve -hf saidutta69/Parable-Granite-4.1-3B-Claude-Fable-5-heretic to pull the default Q4_K_M quant.
bash
# llama.cpp - defaults to Q4_K_Mllama serve -hf saidutta69/Parable-Granite-4.1-3B-Claude-Fable-5-heretic:Q4_K_M# Ollamaollama run hf.co/saidutta69/Parable-Granite-4.1-3B-Claude-Fable-5-heretic:Q4_K_M
python
# transformersfrom transformers import AutoModelForCausalLM, AutoTokenizermodel_id = "saidutta69/Parable-Granite-4.1-3B-Claude-Fable-5-heretic"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id,torch_dtype="auto",device_map="auto",)messages = [{"role": "user", "content": "Write a Python function that retries an HTTP request with exponential backoff."}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=3000, temperature=0.7, top_p=0.95, do_sample=True)print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))# Note: Parable emits <think>...</think> reasoning blocks before the answer (Fable 5 heritage). Strip or surface them as needed.
Also runnable via LM Studio, Jan, vLLM, SGLang — see the "Use this model" widget above for copy-paste commands.
Every answer opens with a <think>...</think> block — that's the Fable 5 trace heritage. Use llama.cpp --jinja to separate it automatically. Sampling: temperature 0.7, top_p 0.95, and budget 2500+ tokens for full reasoning.
Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. You are responsible for how you deploy it — don't put this behind an unmoderated public endpoint serving third parties.
Inherits the apache-2.0 license from the base model (ibm-granite/granite-4.1-3b) via AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5.
| mlp.down_proj.min_weight_distance | 1.63 |
| 1.57 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q3_K_S.gguf | GGUF Q3_K_S | 1.57 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q3_K_M.gguf | GGUF Q3_K_M | 1.73 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q3_K_L.gguf | GGUF Q3_K_L | 1.86 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-IQ4_XS.gguf | GGUF IQ4_XS | 1.91 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q4_K_S.gguf | GGUF Q4_K_S | 2.00 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q4_0.gguf | GGUF Q4_0 | 1.98 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q4_1.gguf | GGUF Q4_1 | 2.18 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q4_K_M.gguf | GGUF Q4_K_M | 2.10 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q5_K_S.gguf | GGUF Q5_K_S | 2.38 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q5_K_M.gguf | GGUF Q5_K_M | 2.44 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q6_K.gguf | GGUF Q6_K | 2.80 GB |
Parable-Granite-4.1-3B-Claude-Fable-5-heretic-Q8_0.gguf | GGUF Q8_0 | 3.62 GB |
reproduce/ | Config + eval transcripts + checksums | — |