GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
BrokenCompute
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
BrokenCompute
Model Tree
Qwen/Qwen3.5-4B
Input Modalities
Output Modalities
Supported Functionality
Given a question, the model takes one of three actions:
web_search(query) tool and grounds its answer on the result.[IDK] and a brief reason, rather than guessing.Its abstention tracks difficulty: it declines more as questions get harder, and (with search) reserves [IDK] for what genuinely isn't findable.
\boxed{}-style output prompts suppress it.text
You may answer, or decline with [IDK] and a brief reason if you are notconfident. Do not guess.web_search(query) function backed by SearXNG, Serper, or any search API. Without it, the model can only answer or decline.top_k 40, top_p 0.95, min_p 0.05, repeat_penalty 1.1.0.2–1.0 range with little behavioral change, so anything in that band is fine.BF16 for best calibration; Q4_K_M (~2.7 GB) runs on 8 GB GPUs and abstains slightly more.Formats in this repo:
safetensors (merged bf16, ~8 GB) — for 🤗 Transformers / vLLM; load by repo id (see below).IDK-v1-Q4_K_M.gguf (~2.7 GB) — runs on 8 GB GPUs; llama.cpp / LM Studio.IDK-v1-BF16.gguf (~8.4 GB) — full precision for llama.cpp / LM Studio.The base is a vision-language model, so load with AutoModelForImageTextToText (not AutoModelForCausalLM), and apply the chat template with thinking on:
python
import torchfrom transformers import AutoModelForImageTextToText, AutoTokenizer tok = AutoTokenizer.from_pretrained("BrokenCompute/IDK-v1")model = AutoModelForImageTextToText.from_pretrained( "BrokenCompute/IDK-v1", dtype=torch.bfloat16, device_map="auto") msgs = [ {"role": "system", "content": "You may answer, or decline with [IDK] and a brief reason if you are not confident. Do not guess."}, {"role": "user", "content": "Which jurist said the First Amendment 'may finally have worked itself pure'?"},]text = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False, enable_thinking=True)out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=512)print(tok.decode(out[0], skip_special_tokens=True))web_search tool use.Q4_K_M tracks BF16 closely on most benchmarks.Built on Qwen3.5-4B (Apache-2.0). This fine-tune and model card are released under Apache-2.0.
import torchfrom transformers import AutoModelForImageTextToText, AutoTokenizer tok = AutoTokenizer.from_pretrained("BrokenCompute/IDK-v1")model = AutoModelForImageTextToText.from_pretrained( "BrokenCompute/IDK-v1", dtype=torch.bfloat16, device_map="auto") msgs = [ {"role": "system", "content": "You may answer, or decline with [IDK] and a brief reason if you are not confident. Do not guess."}, {"role": "user", "content": "Which jurist said the First Amendment 'may finally have worked itself pure'?"},]text = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False, enable_thinking=True)out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=512)print(tok.decode(out[0], skip_special_tokens=True))