GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
saidutta69
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
saidutta69
Model Tree
XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
Input Modalities
Output Modalities
Supported Functionality
Full GGUF ladder included - pick the quant that fits your card:
| Your GPU | Recommended quant | Weights |
|---|---|---|
| RTX 4090 / 5090 (24 GB) | Q8_0 | 8.87 GB |
| RTX 4080 / 5080 / 4060 Ti 16G (16 GB) | Q6_K | 6.85 GB |
| RTX 3060 / 4070 / 5070 (12 GB) | Q5_K_M | 6.02 GB |
| RTX 4060 / 3070 (8 GB) | Q4_K_M | 5.24 GB |
| GTX 1660 Super / 2060 / 3050 laptop (6 GB) | IQ4_XS | 4.88 GB |
| CPU-only / Apple Silicon | Q4_K_M | 5.24 GB |
Weights only, at this model's native 9B size; add ~1 GB per 32K of context. OOM? Drop one quant level. Headroom to spare? Go one up.
Trial 109 of a 200-trial Heretic run (seed 1903262967).
| Parameter | Value |
|---|---|
| direction_index | 23.77 |
| attn.o_proj.max_weight | 1.44 |
| attn.o_proj.max_weight_position | 22.49 |
| attn.o_proj.min_weight | 1.42 |
| attn.o_proj.min_weight_distance | 18.49 |
| mlp.down_proj.max_weight | 1.23 |
| mlp.down_proj.max_weight_position | 23.96 |
| mlp.down_proj.min_weight | 1.05 |
| Metric | This model | Original model (XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B) |
|---|---|---|
| KL divergence | 0.0477 | 0 (by definition) |
| Refusals | 8/100 | 99/100 |
KL divergence of 0.0477 combined with refusals at 8/100 makes this the best refusal-removal /
fidelity trade-off in this batch: the harmful evaluation set drops from 99/100 to 8/100 while the
SFT'd agentic behaviour stays close to the original. Note direction_index is a single index
(23.77) here rather than per-layer, which is why the edit lands tighter than the per-layer runs.
Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the Heretic repo and the original abliteration writeup for the mechanism.
Made with ❤️ by RACER IS OP — follow for more uncensored models
| File | Size |
|---|---|
model-00001-of-00004.safetensors | 4.60 GB |
model-00002-of-00004.safetensors | 4.65 GB |
model-00003-of-00004.safetensors | 4.61 GB |
model-00004-of-00004.safetensors | 3.66 GB |
BF16, ~9B. The reproduce/ directory carries the full Heretic recipe -
config.toml, requirements.txt, the Optuna study journal, and SHA-256 sums - so this exact
model can be regenerated bit-for-bit. Reproduce it with heretic --reproduce reproduce/reproduce.json.
Full quantization set (F16 + Q4_K_M, Q5_K_M, Q6_K, Q8_0) produced with llama.cpp.
| File | Format | Size |
|---|---|---|
MiMo-V2.6-Distill-Qwen-9B-heretic-F16.gguf | GGUF F16 | 16.69 GB |
MiMo-V2.6-Distill-Qwen-9B-heretic-Q4_K_M.gguf | GGUF Q4_K_M | 5.24 GB |
MiMo-V2.6-Distill-Qwen-9B-heretic-Q5_K_M.gguf | GGUF Q5_K_M | 6.02 GB |
MiMo-V2.6-Distill-Qwen-9B-heretic-Q6_K.gguf | GGUF Q6_K | 6.85 GB |
Qwen3.5 architecture (qwen35) - loads natively in llama.cpp / Ollama / LM Studio / Jan.
Run llama serve -hf saidutta69/MiMo-V2.6-Distill-Qwen-9B-heretic to pull the default quant.
bash
# llama.cppllama serve -hf saidutta69/MiMo-V2.6-Distill-Qwen-9B-hereticpython
# transformersfrom transformers import AutoProcessor, AutoModelForImageTextToText model_name = "saidutta69/MiMo-V2.6-Distill-Qwen-9B-heretic"model = AutoModelForImageTextToText.from_pretrained(model_name, dtype="auto", device_map="auto")processor = AutoProcessor.from_pretrained(model_name) messages = [{"role": "user", "content": "Find the bug in this function and rewrite it:\n\ndef avg(xs):\n return sum(xs)/len(xs)+1"}]inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=2048)print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))Also runnable via Ollama, LM Studio, Jan, vLLM, SGLang.
The model keeps its vision encoder, so you can hand it a screenshot and ask for the code that produced the UI:
python
messages = [{"role": "user", "content": [ {"type": "image", "image": "https://example.com/screenshot.png"}, {"type": "text", "text": "Write the React and Tailwind that reproduces this layout."},]}]Pass tool definitions straight through apply_chat_template — MiMo-V2.6 was SFT'd for tool calling,
so the format is already in-distribution and does not need adapting.
| Architecture | Qwen3_5ForConditionalGeneration (hybrid linear-attention + full-attention, multimodal) |
| Parameters | ~9B |
| Layers / heads | 32 layers (24 linear-attention + 8 full-attention every 4th), 16 attention heads, 4 KV heads, head dim 256 |
| Hidden / intermediate | 4096 / 12288 |
| Position embedding | mRoPE (interleaved, sections 11/11/10), partial rotary 0.25, theta = 10,000,000 |
| Context length | 262,144 |
| Vocab | 248,320 |
As reported in the MiMo-V2.6 technical report for the released SFT checkpoint:
| Domain | Benchmark | Metric | Qwen3.5-9B | MiMo-V2.6-Distill-Qwen-9B (SFT) |
|---|---|---|---|---|
| Code | SWE Verified | avg@3 | 60.0 | 61.1 |
Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. This model is explicitly tuned for cybersecurity work, so take that seriously: you are responsible for how you deploy it and for what you point it at. It inherits MiMo-V2.6's factual limitations and biases; abliteration removes refusal directions, it doesn't add capability or judgment.
Inherits the mit license from the base model.
# transformersfrom transformers import AutoProcessor, AutoModelForImageTextToText model_name = "saidutta69/MiMo-V2.6-Distill-Qwen-9B-heretic"model = AutoModelForImageTextToText.from_pretrained(model_name, dtype="auto", device_map="auto")processor = AutoProcessor.from_pretrained(model_name) messages = [{"role": "user", "content": "Find the bug in this function and rewrite it:\n\ndef avg(xs):\n return sum(xs)/len(xs)+1"}]inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=2048)print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))messages = [{"role": "user", "content": [ {"type": "image", "image": "https://example.com/screenshot.png"}, {"type": "text", "text": "Write the React and Tailwind that reproduces this layout."},]}]| mlp.down_proj.min_weight_distance | 11.59 |
MiMo-V2.6-Distill-Qwen-9B-heretic-Q8_0.gguf | GGUF Q8_0 | 8.87 GB |
| Precision | bfloat16 |
| Base model | XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B, SFT'd from Qwen/Qwen3.5-9B |