GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ninhnv
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ninhnv
Model Tree
google/gemma-4-E2B-it
Input Modalities
Output Modalities
Supported Functionality
bash
pip install "chon[serve] @ git+https://github.com/xoai/chon"python -m chon.serve --model ninhnv/chon-1.0-e2b --port 8000 # downloads the base model and this adapterOn a Mac with Apple silicon (uses the Mac's GPU), or on CPU only:
bash
python -m chon.serve --model ninhnv/chon-1.0-e2b --device mps --port 8000python -m chon.serve --model ninhnv/chon-1.0-e2b --device cpu --port 8000bash
curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ "state": "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.", "questions": {"decision": {"type": "choice", "instructions": "Route this ticket.", "criteria": {"refund": "Refund or replacement", "shipping": "Delivery status", "other": "Anything else"}}}}'The response holds answers.decision with choice and probabilities for choice, noul (P(yes)) for
yes/no, or probabilities over the levels for score. usage reports input_tokens and output_tokens; every decision is a single forward pass, so output_tokens is 0.
| File | What it is |
|---|---|
adapter_model.safetensors, adapter_config.json | the LoRA adapter (PEFT) |
chon_config.json | serving settings (prompt and calibration) read by the chon server |
| tokenizer files | the base model's tokenizer and chat template, unchanged |
README.md | this card |
| set | Intelligence | Calibration (proxy) | Capability (proxy) |
|---|---|---|---|
| held-out replica (720) | 57.6 | 79.7 | 68.7 |
| JevBench public (231) | 70.3 | 85.0 | 77.6 |
Serving, measured on one H100 80 GB, one request at a time: median response 0.05 s, p95 0.1 s; 740 input and 0 output tokens per decision on average.
On a MacBook Pro (Apple M3 Pro, 18 GB) with --device mps, bf16, one request at a time: median response 0.96 s, p95 2.2 s over 60 random held-out questions (810 input tokens on average).
On CPU only (bf16, one request at a time): median 1.8 s on 8 threads of an Intel Xeon Platinum 8480+ and 5.8 s on an Apple M3 Pro's 5 performance cores, over 30 short held-out questions (about 230 input tokens; typical questions are about three times longer and take roughly proportionally longer). On a Mac, use --device mps.
Official JevBench harness (typesafe adapter, public files) against the served package, accuracy: easy 100.0%, original 94.4%, hard 73.0%.
Apache-2.0, as google/gemma-4-E2B-it.
pip install "chon[serve] @ git+https://github.com/xoai/chon"python -m chon.serve --model ninhnv/chon-1.0-e2b --port 8000 # downloads the base model and this adapterpython -m chon.serve --model ninhnv/chon-1.0-e2b --device mps --port 8000python -m chon.serve --model ninhnv/chon-1.0-e2b --device cpu --port 8000curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ "state": "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.", "questions": {"decision": {"type": "choice", "instructions": "Route this ticket.", "criteria": {"refund": "Refund or replacement", "shipping": "Delivery status", "other": "Anything else"}}}}'