GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ninhnv
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ninhnv
Model Tree
google/gemma-4-12B-it
Input Modalities
Output Modalities
Supported Functionality
bash
pip install "chon[serve] @ git+https://github.com/xoai/chon@v1.1.1"python -m chon.serve --model ninhnv/chon-1.0-12b-fast --port 8000 # downloads the base model and this adapterThe first start downloads the base model, google/gemma-4-12B-it (public, no token needed), into the Hugging Face cache. --device defaults to auto: an NVIDIA GPU if there is one, else Apple's GPU, else the CPU.
bash
curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ "state": "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.", "questions": {"decision": {"type": "choice", "instructions": "Route this ticket.", "criteria": {"refund": "Refund or replacement", "shipping": "Delivery status", "other": "Anything else"}}}}'The response holds answers.decision with choice and probabilities for choice, noul (P(yes); also as probabilities keyed true / false) for
yes/no, or probabilities over the levels for score. usage reports input_tokens and output_tokens; every decision is a single forward pass, so output_tokens is 0.
python
from chon.infer import DecisionModel dm = DecisionModel.load("ninhnv/chon-1.0-12b-fast", device="auto") # NVIDIA GPU, else Apple's GPU, else CPUanswers, usage = dm.predict( "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.", {"damaged": {"type": "noul", "instructions": "Did the order arrive damaged?"}})print(answers["damaged"]["noul"]) # P(yes)When option labels are only ids ("0", "1", ...), add "option_keys": "hide" to the question so the model sees only
the option descriptions.
| File | What it is |
|---|---|
adapter_model.safetensors, adapter_config.json | the LoRA adapter (PEFT) |
chon_config.json | serving settings (prompt and calibration) read by the chon server |
| tokenizer files | the base model's tokenizer and chat template, unchanged |
README.md | this card |
| set | Intelligence | Calibration (proxy) | Capability (proxy) |
|---|---|---|---|
| held-out replica (720) | 72.4 | 86.5 | 79.4 |
| JevBench public (231) | 78.8 | 87.8 | 83.3 |
Serving, measured on one H100 80 GB, one request at a time: median response 0.07 s, p95 0.1 s; 744 input and 0 output tokens per decision on average, about 0.037per1,000decisionsat0.05 / $0.1 per million input / output tokens.
Apache-2.0, as google/gemma-4-12B-it.
pip install "chon[serve] @ git+https://github.com/xoai/chon@v1.1.1"python -m chon.serve --model ninhnv/chon-1.0-12b-fast --port 8000 # downloads the base model and this adaptercurl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ "state": "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.", "questions": {"decision": {"type": "choice", "instructions": "Route this ticket.", "criteria": {"refund": "Refund or replacement", "shipping": "Delivery status", "other": "Anything else"}}}}'from chon.infer import DecisionModel dm = DecisionModel.load("ninhnv/chon-1.0-12b-fast", device="auto") # NVIDIA GPU, else Apple's GPU, else CPUanswers, usage = dm.predict( "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.", {"damaged": {"type": "noul", "instructions": "Did the order arrive damaged?"}})print(answers["damaged"]["noul"]) # P(yes)