GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
sjoe1244
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
sjoe1244
Model Tree
coder3101/gemma-4-31B-it-heretic
Input Modalities
Output Modalities
Supported Functionality
| File | Notes |
|---|---|
model-00001-of-00003.safetensors … 00003 | EXL3 weights + BF16 vision |
quantization_config.json | EXL3 4.00 bpw, head_bits 6, codebook mcg |
preprocessor_config.json / processor_config.json | Gemma 4 vision processor |
tokenizer.json | tokenizer; no chat_template in tokenizer_config.json |
gemma4.jinja | Gemma 4 chat template to point TabbyAPI at |
Do not use this folder as a Transformers / bitsandbytes load. Load it with
ExLlamaV3 or TabbyAPI's exllamav3 loader.
From quantization_config.json:
0.0.29Converted locally on an NVIDIA RTX 4090 (August 2026).
Worked here in TabbyAPI:
cache_mode: Q8cache_size: 8192vision: true (or omit; vision tensors are in the shards either way)chunk_size: 2048prompt_template at gemma4.jinja if the tokenizer has no chat templateRough VRAM with vision + Q8 8K KV: ~18.6 GB.
Gemma 4 terms apply to the original Google weights. Heretic / abliteration
credit belongs to coder3101.
This repo is only the EXL3 export.