GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
saidutta69
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
saidutta69
Model Tree
Qwen/Qwen3-4B-Thinking-2507
Input Modalities
Output Modalities
Supported Functionality
Full default quant set - pick the one that fits your card:
| Your GPU | Recommended quant | Weights |
|---|---|---|
| RTX 3090 / 4090 / 5090 (24 GB) | Q8_0 | 4.28 GB |
| RTX 4080 / 5080 / 4060 Ti 16G (16 GB) | Q6_K | 3.31 GB |
| RTX 3060 / 4070 / 5070 (12 GB) | Q5_K_M | 2.89 GB |
| RTX 4060 / 3070 (8 GB) | Q4_K_M | 2.50 GB |
| CPU-only / Apple Silicon | Q4_K_M | fits in system RAM |
Weights only, at this model's 4.02B native size. Thinking mode writes a lot of tokens before it answers, so budget real KV-cache headroom on top - a reasoning trace of a few thousand tokens costs more than the weights do on a long conversation. OOM? Drop one quant level.
Trial 88 of a 200-trial Heretic run (seed 263519039).
| Parameter | Value |
|---|---|
| direction_index | 17.68 |
| attn.o_proj.max_weight | 1.45 |
| attn.o_proj.max_weight_position | 21.22 |
| attn.o_proj.min_weight | 1.19 |
| attn.o_proj.min_weight_distance | 19.74 |
| mlp.down_proj.max_weight | 1.36 |
| mlp.down_proj.max_weight_position | 22.21 |
| mlp.down_proj.min_weight | 0.21 |
| Metric | This model | Original model (Qwen/Qwen3-4B-Thinking-2507) |
|---|---|---|
| KL divergence | 0.0011 | 0 (by definition) |
| Refusals | 4/100 | 98/100 |
This is about as clean as abliteration gets. Refusals on the harmful evaluation set drop from 98/100 to 4/100 - the base model was almost fully refusal-tuned, and nearly all of it is gone - while the KL divergence of 0.0011 is the tightest in this collection, meaning the weight edit barely perturbed the network. Both at once is the point: the refusal behaviour here was a cleanly separable direction, so removing it cost almost nothing.
Worth saying plainly, because it is not automatic: a low KL number on its own does not mean a good abliteration. Another model in this collection, FrogNano-4B-heretic, has a similarly tight KL of 0.0046 and still refuses 71/100 times - there the optimiser found a direction it barely dared apply, so it preserved behaviour and preserved most refusals. Judge the pair of numbers together, never the KL alone.
Abliteration does not add capability - these are the base numbers, from Qwen's own eval table
(column Qwen3-4B-Thinking-2507). No post-abliteration eval has been run.
| Task | Benchmark | Score |
|---|---|---|
| Math reasoning | AIME25 | 81.3 |
| Math reasoning | HMMT25 | 55.5 |
| General | LiveBench (20241125) | 71.8 |
| Knowledge | MMLU-Pro | 74.0 |
| Knowledge | GPQA | 65.8 |
| Code | LiveCodeBench v6 |
Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the Heretic repo and the original abliteration writeup for the mechanism.
Made with ❤️ by RACER IS OP — follow for more uncensored models
| File | Size |
|---|---|
model-00001-of-00002.safetensors | 4.99 GB |
model-00002-of-00002.safetensors | 3.05 GB |
BF16, 4,022,468,096 parameters, with tied input/output embeddings. The reproduce/ directory carries
the full Heretic recipe - config.toml, requirements.txt, the Optuna study journal, and SHA-256
sums - so this exact model can be regenerated bit-for-bit. Reproduce it with
heretic --reproduce reproduce/reproduce.json.
Default set (F16 + 4 quants) produced with llama.cpp.
| File | Format | Size |
|---|---|---|
Qwen3-4B-Thinking-heretic-F16.gguf | GGUF F16 | 8.05 GB |
Qwen3-4B-Thinking-heretic-Q4_K_M.gguf | GGUF Q4_K_M | 2.50 GB |
Qwen3-4B-Thinking-heretic-Q5_K_M.gguf | GGUF Q5_K_M | 2.89 GB |
Qwen3-4B-Thinking-heretic-Q6_K.gguf | GGUF Q6_K | 3.31 GB |
Qwen3 dense architecture (qwen3, 398 tensors, 36 layers) - loads
natively in llama.cpp / Ollama / LM Studio / Jan.
Run llama serve -hf saidutta69/Qwen3-4B-Thinking-heretic to pull the default quant.
bash
# llama.cppllama serve -hf saidutta69/Qwen3-4B-Thinking-hereticpython
# transformers (requires transformers>=5.0.0)from transformers import AutoModelForCausalLM, AutoTokenizer model_name = "saidutta69/Qwen3-4B-Thinking-heretic"model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")tokenizer = AutoTokenizer.from_pretrained(model_name) messages = [{"role": "user", "content": "A train leaves at 14:05 travelling 80 km/h. A car leaves at 14:15 travelling 120 km/h on the same track. When does the car catch the train? Show your work."}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, temperature=0.6, top_p=0.95, top_k=20, max_new_tokens=4096)print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))This checkpoint always thinks. There is no enable_thinking flag to turn it off - the chat
template appends <think> to the generation prompt unconditionally, because the Thinking-2507
release dropped the hybrid thinking/non-thinking toggle that
Qwen3-4B-Instruct-2507-heretic
still exposes. If you need to disable thinking, use that model instead.
Parsing gotcha: it is normal for the output to contain only </think> with no opening
<think> tag, because the template already supplied the opening half. Strip reasoning with a
regex that tolerates a missing opening tag:
python
import redef split_think(text): m = re.search(r"</think>(.*)", text, re.S) return (m.group(1).strip() if m else ""), (re.sub(r"^.*?</think>", "", text, flags=re.S).strip() if "</think>" in text else "")For multi-turn conversations, pass the previous turn's reasoning back through
reasoning_content on the assistant message; the template reconstructs the
<think>...</think> block for you.
Qwen's recommended settings for thinking mode, which the repo's generation_config.json already
sets, so generate() picks them up with no arguments:
python
temperature=0.6, top_p=0.95, top_k=20, do_sample=TrueStop tokens are <|im_end|> (151645) and <|endoftext|> (151643) - both are already configured.
Do not pass enable_thinking; this template does not accept it.
Pass OpenAI-style function definitions as tools. The template injects the <tools> system block
and instructs the model to emit calls as JSON inside <tool_call> tags; feed results back under a
tool role and the template wraps them in <tool_response> for you. Thinking happens before the
tool call, so budget for a reasoning trace per turn.
| Architecture | Qwen3ForCausalLM (decoder-only dense transformer) |
| Parameters | 4.02B |
| Layers / heads | 36 layers (all full attention), 32 attention heads, 8 KV heads (GQA), head_dim 128 |
| Hidden / intermediate | 2560 / 9728 (SwiGLU) |
| Position embedding | RoPE, theta = 5,000,000 |
| Context length | 262,144 native |
| Vocab | 151,936 |
Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. You are responsible for how you deploy it - don't put this behind an unmoderated public-facing endpoint serving third parties. It inherits Qwen3's factual limitations and biases; abliteration removes refusal directions, it doesn't add capability or judgment.
Because thinking mode narrates its reasoning before acting, be aware that the visible chain-of-thought is a reasoning artefact, not a reliable audit trail - a model can produce a persuasive trace that does not match how it actually reached the answer.
Inherits the apache-2.0 license from the base
model, which ships a LICENSE file.
enable_thinking toggle| mlp.down_proj.min_weight_distance | 18.62 |
| 55.2 |
| Code | CFEval | 1852 |
| Instruction following | IFEval | 87.4 |
| Alignment / preference | Arena-Hard v2 | 34.9 |
| Alignment / writing | WritingBench | 83.3 |
| Agent / tool calling | BFCL-v3 | 71.2 |
| Agent | TAU2-Retail | 53.5 |
| Agent | TAU2-Airline | 58.0 |
Qwen3-4B-Thinking-heretic-Q8_0.gguf | GGUF Q8_0 | 4.28 GB |
| bfloat16, tied embeddings |
| Reasoning | always-on <think> chain-of-thought, no off switch |
| Languages | English, Chinese |
| Base model | Qwen/Qwen3-4B-Thinking-2507 |