GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ninhnv
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ninhnv
Model Tree
google/gemma-4-E4B-it
Input Modalities
Output Modalities
Supported Functionality
bash
pip install "chon[serve] @ git+https://github.com/xoai/chon"python -m chon.serve --model ninhnv/chon-1.0-e4b --port 8000 # downloads the base model and this adapterOn a Mac with Apple silicon and 24 GB or more (uses the Mac's GPU), or on CPU only:
bash
python -m chon.serve --model ninhnv/chon-1.0-e4b --device mps --no-think --port 8000python -m chon.serve --model ninhnv/chon-1.0-e4b --device cpu --no-think --port 8000--no-think makes every decision a single forward pass. Use it on a Mac or CPU: the short extra pass for unsure questions generates text, which is slow without a data-centre GPU. Chon-1.0-E2B is the better fit for Macs with less memory.
bash
curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ "state": "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.", "questions": {"decision": {"type": "choice", "instructions": "Route this ticket.", "criteria": {"refund": "Refund or replacement", "shipping": "Delivery status", "other": "Anything else"}}}}'The response holds answers.decision with choice and probabilities for choice, noul (P(yes)) for
yes/no, or probabilities over the levels for score. usage reports input_tokens and output_tokens: most decisions are a single forward pass with no output tokens, while some take a short extra pass whose generated tokens are counted in output_tokens.
| File | What it is |
|---|---|
adapter_model.safetensors, adapter_config.json | the LoRA adapter (PEFT) |
chon_config.json | serving settings (prompt and calibration) read by the chon server |
| tokenizer files | the base model's tokenizer and chat template, unchanged |
README.md | this card |
| set | Intelligence | Calibration (proxy) | Capability (proxy) |
|---|---|---|---|
| held-out replica (720) | 72.0 | 83.8 | 77.9 |
| JevBench public (231) | 77.9 | 85.7 | 81.8 |
Serving, measured on one H100 80 GB, one request at a time: median response 0.08 s, p95 21.4 s; 740 input and 121 output tokens per decision on average.
On CPU only (8 threads of an Intel Xeon Platinum 8480+, bf16, one request at a time, --no-think): median response 3.0 s over 30 short held-out questions (about 230 input tokens; typical questions are about three times longer and take roughly proportionally longer). On a MacBook Pro with 18 GB (Apple M3 Pro) the model did not fit in memory alongside the system; use a Mac with 24 GB or more.
Official JevBench harness (typesafe adapter, public files) against the served package, accuracy: easy 100.0%, original 98.6%, hard 78.4%.
Apache-2.0, as google/gemma-4-E4B-it.
pip install "chon[serve] @ git+https://github.com/xoai/chon"python -m chon.serve --model ninhnv/chon-1.0-e4b --port 8000 # downloads the base model and this adapterpython -m chon.serve --model ninhnv/chon-1.0-e4b --device mps --no-think --port 8000python -m chon.serve --model ninhnv/chon-1.0-e4b --device cpu --no-think --port 8000curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ "state": "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.", "questions": {"decision": {"type": "choice", "instructions": "Route this ticket.", "criteria": {"refund": "Refund or replacement", "shipping": "Delivery status", "other": "Anything else"}}}}'