GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
saidutta69
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
saidutta69
Model Tree
HuggingFaceTB/SmolLM2-1.7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Device | Recommended quant | Weights |
|---|---|---|
| Phone / browser (6 GB RAM) | Q4_K_M | 1.06 GB |
| Raspberry Pi 5 (8 GB) | Q5_K_M | 1.23 GB |
| Laptop (8 GB, Apple Silicon) | Q8_0 | 1.82 GB |
| Server / multi-tenant | Q6_K | 1.41 GB |
Context is the real cost here: this model's native window is 8,192 tokens, and KV cache runs roughly 16 MB per 8K of context at Q4. The Q2_K tier (~0.67 GB) is there if you are truly RAM-bound, but at 1.7B the model still needs about 1 GB of weights before any cache.
Trial of a 200-trial Heretic run (seed 177211886), direction_index selected per layer.
| Parameter | Value |
|---|---|
| direction_index | 15.66 |
| attn.o_proj.max_weight | 1.37 |
| attn.o_proj.max_weight_position | 17.70 |
| attn.o_proj.min_weight | 0.60 |
| attn.o_proj.min_weight_distance | 7.23 |
| mlp.down_proj.max_weight | 1.47 |
| mlp.down_proj.max_weight_position | 18.68 |
| mlp.down_proj.min_weight | 1.46 |
| mlp.down_proj.min_weight_distance | 2.90 |
| Metric | This model | Original model (HuggingFaceTB/SmolLM2-1.7B-Instruct) |
|---|---|---|
| KL divergence | 0.0257 | 0 (by definition) |
| Refusals | 8/100 | 37/100 |
Read those two numbers together, they are only meaningful as a pair. KL 0.0257 is a tight edit for this collection - looser than granite-4.2-3b-heretic (0.0095) and LFM2.5-VL-450M-heretic (0.0187), but far tighter than the high-drift models in it - and it clears 29 of the base model's 37 refusals while leaving the model's capabilities intact.
The base model's own published scores, which the abliteration does not change, are IFEval 56.7, MT-Bench 6.13, GSM8K (5-shot) 48.2, BBH (3-shot) 32.2, MMLU-Pro (MCF) 19.3, ARC 51.7, PIQA 74.4 and HellaSwag 66.1, with a BFCL leaderboard score of 27% for function calling. Those are the base model's numbers, reproduced from the upstream card; no separate benchmark sweep was run on this abliterated checkpoint.
Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the Heretic repo and the original abliteration writeup for the mechanism.
Made with ❤️ by RACER IS OP - follow for more uncensored models
| File | Size |
|---|---|
model.safetensors | 3.42 GB |
BF16, 1.71B parameters (embeddings tied). The reproduce/ directory carries the full
Heretic recipe - config.toml, requirements.txt, the Optuna study journal, and SHA-256 sums - so
this exact model can be regenerated bit-for-bit with heretic --reproduce reproduce/reproduce.json.
Full quantization set (14 quants + F16) produced with llama.cpp.
| File | Format | Size |
|---|---|---|
SmolLM2-1.7B-Instruct-heretic-F16.gguf | GGUF F16 | 3.42 GB |
SmolLM2-1.7B-Instruct-heretic-Q2_K.gguf | GGUF Q2_K | 0.67 GB |
SmolLM2-1.7B-Instruct-heretic-IQ3_S.gguf | GGUF IQ3_S | 0.78 GB |
SmolLM2-1.7B-Instruct-heretic-Q3_K_S.gguf | GGUF Q3_K_S | 0.78 GB |
SmolLM2-1.7B-Instruct-heretic-Q3_K_M.gguf | GGUF Q3_K_M | 0.86 GB |
SmolLM2-1.7B-Instruct-heretic-Q3_K_L.gguf | GGUF Q3_K_L | 0.93 GB |
SmolLM2-1.7B-Instruct-heretic-IQ4_XS.gguf | GGUF IQ4_XS | 0.95 GB |
SmolLM2-1.7B-Instruct-heretic-Q4_K_S.gguf | GGUF Q4_K_S | 1.00 GB |
SmolLM2-1.7B-Instruct-heretic-Q4_0.gguf | GGUF Q4_0 | 0.99 GB |
SmolLM2-1.7B-Instruct-heretic-Q4_1.gguf | GGUF Q4_1 | 1.09 GB |
SmolLM2-1.7B-Instruct-heretic-Q4_K_M.gguf | GGUF Q4_K_M | 1.06 GB |
SmolLM2-1.7B-Instruct-heretic-Q5_K_S.gguf | GGUF Q5_K_S | 1.19 GB |
SmolLM2-1.7B-Instruct-heretic-Q5_K_M.gguf | GGUF Q5_K_M | 1.23 GB |
SmolLM2-1.7B-Instruct-heretic-Q6_K.gguf | GGUF Q6_K | 1.41 GB |
SmolLM2-1.7B-Instruct-heretic-Q8_0.gguf | GGUF Q8_0 | 1.82 GB |
Total GGUF payload: 18.17 GB across 15 files.
Architecture is llama for the text tower - a plain dense Llama, 24 layers, 32 attention heads,
32 KV heads, head dim 64, RoPE theta 130,000, vocab 49,152 - so it loads natively in llama.cpp,
LM Studio, Jan, Ollama, vLLM and SGLang.
bash
# llama.cpp - defaults to the Q4_K_M quantllama-cli -hf saidutta69/SmolLM2-1.7B-Instruct-heretic:Q4_K_M -ngl 99python
# transformersfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "saidutta69/SmolLM2-1.7B-Instruct-heretic"model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")tokenizer = AutoTokenizer.from_pretrained(model_name) messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Summarise the CAP theorem in three sentences."},]inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True).to(model.device)outputs = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.2, top_p=0.9)print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))Requires transformers>=5.20.0 - config.json uses dtype: rather than torch_dtype:, so older
transformers will not load it. Use dtype="auto".
Also runnable via Ollama, LM Studio, Jan, vLLM and SGLang.
| Architecture | LlamaForCausalLM - dense, no MoE, no hybrid layers |
| Parameters | 1.71B (embeddings tied) |
| Layers | 24 |
| Attention | 32 heads, 32 KV heads (no GQA), head dim 64 |
| Hidden / intermediate | 2048 / 8192 |
| Position embedding | RoPE theta = 130,000 |
| Context length | 8,192 |
| Vocab | 49,152 (embeddings tied) |
| GGUF | arch llama, 24 blocks |
| Precision | bfloat16 |
| Base model | HuggingFaceTB/SmolLM2-1.7B-Instruct |
SmolLM2 is Hugging Face's compact model line, built for on-device use. The 1.7B variant was trained on 11 trillion tokens from FineWeb-Edu, DCLM and The Stack plus curated math and code data, then instruction-tuned with SmolTalk and preference-tuned with DPO on UltraFeedback. It supports text rewriting, summarization and function calling, and the whole family is designed to stay light enough to run locally on a laptop, a phone or a Pi.
Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. You are responsible for how you deploy it - don't put this behind an unmoderated public-facing endpoint serving third parties. It inherits SmolLM2's factual limitations and biases; abliteration removes refusal directions, it doesn't add capability or judgment.
apache-2.0, inherited from the base model.
Recorded in reproduce/reproduce.json:
Linux-6.8.0-147-generic-x86_64-with-glibc2.35Heretic notes that KL-divergence and refusal numbers can vary slightly with hardware and library versions.
# llama.cpp - defaults to the Q4_K_M quantllama-cli -hf saidutta69/SmolLM2-1.7B-Instruct-heretic:Q4_K_M -ngl 99# transformersfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "saidutta69/SmolLM2-1.7B-Instruct-heretic"model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")tokenizer = AutoTokenizer.from_pretrained(model_name) messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Summarise the CAP theorem in three sentences."},]inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True).to(model.device)outputs = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.2, top_p=0.9)print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))