GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
hilman2
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
hilman2
Model Tree
google/gemma-4-E4B-it-qat-q4_0-unquantized
Input Modalities
Output Modalities
Supported Functionality
| File | Use |
|---|---|
model.safetensors, config.json, tokenizer files | the weights, bf16, for building a Mev engine on a Mac (scripts/build-engine.sh gev-e4b) or for your own tools |
gev-e4b-q8_0.gguf | for llama.cpp, 8.0 GB; Gev's Docker image downloads it |
gev-e4b-q4_0.gguf | for llama.cpp on smaller GPUs, 5.2 GB, laid out like Google's QAT Q4_0 GGUF |
Retraining moved the weights off the grid Google's quantization-aware training had put them on, so Q4_0 costs this model more than it costs the original. On the 316 example requests of Gev's playground, the most probable answer through llama.cpp matched the one from the unquantized weights for 99.1 % of the questions with Q8_0 and 93.9 % with Q4_0 (Google's E4B at Q4_0: 96.2 %).
A ready-built engine for Apple's M4 is in hilman2/gev-e4b-mev.
Measured on a Mac mini with an M4 Pro and 48 GB, through Gev's Mev engine.
| Gev E4B | Gemma 4 E4B | Gemma 4 26B-A4B | |
|---|---|---|---|
| Correct decisions in 7,671 test cases | 88.8 % | 83.6 % | 88.2 % |
| Requests per second, 8 at a time | 4.54 | 2.52 |
Apache License 2.0, as Gemma 4.