GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
frontier-infra
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
frontier-infra
Model Tree
Qwen/Qwen3.5-4B-Base
Input Modalities
Output Modalities
Supported Functionality
Measured by us with AINode's bench, one logit read per question, the same rendered prompt for every model. Accuracy is the share of questions whose top label is the human label. Decision Score is Jevals' metric: 100 = perfect, 0 = guessing the label base rates, below 0 = worse than that. These are our numbers on the public suites, not rows on the Jevals board. Headline (macro over the zero-shot public sets): 70.3 (v0: 70.9).
| Set (questions) | Type | Accuracy | Floor (majority label) | Decision Score (Jevals) | ECE, temperatures applied | Flips over identical repeats |
|---|---|---|---|---|---|---|
| Jevals PubMedQA (300) | noul | 86.7 | 62.0 | 59.0 | 0.039 | 0.3% |
| Jevals Banking77 (300, 77 options) | choice | 68.3 | 1.3 | 55.6 | 0.063 | 0.0% |
| Jevals HelpSteer2 helpfulness (300, 5 levels) | score | 37.0 | 41.7 | 9.2 | 0.044 (raw 0.073) | 1.0% |
| Nimble held-out eval (324) | mixed | 71.9 | 17.6 | 57.5 | 0.058 | 0.0% |
| Kev transfer-v4 test (764) | mixed | 82.2 | 21.5 | 64.7 | 0.041 | 0.3% |
| Kev decision-v7 test (1,440) | mixed | 78.8 | 20.3 | 67.9 | 0.042 | n/a |
| typed-decisions test (2,000) | mixed | 79.8 | 15.3 | 56.6 | 0.170 | 0.1% |
Nimble's 13 public human-labelled subsets (3,880 questions), macro accuracy: 75.9 (per subset: aegis2 80.0, boolq 85.7, civil_comments 84.0, helpsteer2 38.2, massive-de-DE 83.7, massive-en-US 83.7, multinli 86.0, paws 82.8, pubmedqa 70.8, squad2 79.6, summeval-consistency 86.1, summeval-relevance 51.2, vitaminc-dev 75.3). For scale, Bespoke's published table puts Nimble-9B at 74.8 and Jev at 76.0 on the same subsets with their scorer.
Two things to read carefully. HelpSteer2 and SummEval are no longer zero-shot for Jeb: v1 trained on the HelpSteer2 train split (the Jevals and Nimble items come from validation, and no item overlaps) and on SummEval articles that Nimble does not score (its 15 evaluation articles are held out entirely), so those rows are held-out items of a seen rubric. And the in-distribution score set (typed-decisions) gives back a few points of Decision Score against v0, the cost of honest calibration on human rubrics.
One honest regression on the 4B: the Nimble 324 held-out set fell from 77.5 (v0) to 71.9 with the rubric data in the pool, while the 9B gained on it. The smaller model does not absorb 45 percent score questions without its choice and yes/no judgments slipping; v0 stays the better general 4B, v1 the better calibrated one. A lower rubric share or a higher rank is the next probe.
Every eval record (per question: option keys, probabilities, pick, label, repeat, option order) is in
eval/, with eval/results.json carrying the full metric set.
Every question on the four head sets was scored once clean and three more times with a fresh UUID planted
in the state (state) or appended to the instructions (instr). Records are in eval/nonce/.
| Set | Pick agreement (state / instr) | Questions with any flip (state / instr) | Mean change of p_max, points (state / instr) |
|---|---|---|---|
| Jevals PubMedQA | 98.2% / 98.7% | 3.0% / 2.3% | 1.1 / 1.0 |
| Jevals Banking77 | 96.4% / 98.2% | 4.3% / 3.0% | 2.4 / 2.1 |
| Jevals HelpSteer2 | 92.1% / 95.4% | 10.7% / 7.3% | 1.1 / 0.6 |
| Nimble 324 | 100.0% / 99.0% | 0.0% / 1.5% |
Choice and yes/no answers are close to immune. On score questions v1 flips its top level more often than
v0 while moving its probabilities far less: a calibrated rubric model sits honestly between adjacent
levels, so a small nudge tips the argmax. Read the expected level and the probabilities (which is what the
route returns as score), not the top pick, for rubric questions.
Which temperature fit, and why. temperatures.json carries two fits on the calibration split (689 records, 921 questions: choice 225, noul 147, score 549). The hard fit minimises NLL against the argmax label and sharpens (T 0.80 / 0.97 / 0.79). The train fit minimises NLL against the training target, the soft or ordinal distribution the model was taught, and softens (T 1.20 / 1.33 / 1.16). v1 applies train, because the ordinal target is what makes rubric probabilities honest, and the hard fit is exactly the sharpening that made v0's HelpSteer2 calibration worse. The price is visible in the same file: on the calibration split measured against hard labels, ECE moves from 0.094 to 0.100 (choice), 0.082 to 0.108 (noul) and 0.091 to 0.100 (score) while NLL on the training target falls. The ECE column in the results table is measured on the evaluation sets, not on this split, with the applied temperatures. Both fits are in the file; a consumer who gates on the argmax may prefer hard, and either way should refit on its own data.
The prompt is AINode's own decide rendering (ainode.api.decide.build_messages, source commit
e5c08938, prompt_source_sha256 in prompt_contract.json), through the base chat template with
thinking off. The option labels are single tokens; the model's answer is the distribution over those label
tokens at the last prompt position, read in fp32 from the last hidden state and then temperature scaled
per type (temperatures.json: choice 1.20, noul 1.33, score 1.16). Nothing is generated.
The supported way to run it is AINode, which renders the prompt exactly as trained (a worked
/v1/systemone request and response is in the AINode docs issue tracker until the page lands):
bash
curl -sS https://<your-ainode>/v1/systemone -H "Authorization: Bearer <key>" -H "Content-Type: application/json" -d '{ "model": "frontier-infra/jebadiah-4b-v1", "state": {"ticket": "Customer says the invoice total does not match the quote."}, "questions": { "route": {"type": "choice", "instructions": "Which team should take this ticket?", "criteria": {"billing": "an invoice, a charge or a refund", "support": "a product question", "sales": "a quote or a renewal"}}, "urgent": {"type": "noul", "instructions": "The customer is blocked from working.", "criteria": {"true": "work has stopped", "false": "it can wait"}} }}'scripts/ in this repository carries the trainer's own copies of the renderer and the logit read: ainode_prompt_verbatim.py is AINode's decide rendering copied verbatim from getainode/ainode at e5c08938 (its sha256 is the prompt_source_sha256 in prompt_contract.json), jebadiah_prompt.py turns a /v1/systemone question into that prompt with single-token labels, jebadiah_model.py reads the label-token logits at the answer position in fp32, and decide_standalone.py runs one request end to end:
bash
pip install torch transformers==5.17.0 peft==0.21.0 huggingface_hubhuggingface-cli download frontier-infra/jebadiah-4b-v1 --local-dir jebpython jeb/scripts/decide_standalone.py --base Qwen/Qwen3.5-4B-Base --revision 1001bb4d826a52d1f399e183466143f4da7b741b \ --adapter jeb --request jeb/scripts/example-request.json # add --no-temperatures for the raw distributionIt prints the answers in the route's shape (choice: choice, confidence, probabilities; noul: noul = P(true); score: score, legend, probabilities) plus the temperatures it applied. Runs on CPU too (slow, minutes per request without the DeltaNet kernels), which is how it was checked before publishing.
Thresholds belong to the caller: act on a high probability, confirm or escalate on a middle one, hand a low one to a person or a bigger model. The model never refuses; policy is built from decisions.
LocalLLaMA/typed-decisions train (Apache-2.0,
revision ea930645); the Kev v7 training sources whose licenses permit derived weights (BoolQ, MNLI,
DBpedia14 and Kev's contrastive and composition sets, Kev suites revision a88f56db); nvidia/HelpSteer2
train split (CC BY 4.0, 2,573 score questions: helpfulness in the Jevals task's exact shape for 1,609
rows, plus correctness, coherence, complexity and verbosity on 241 of them); SummEval (MIT, via
mteb/summeval, 1,664 score questions over four dimensions on 26 articles, with Nimble's 15 evaluation
articles excluded). 95/5 family split into 14,892 training records and 689 calibration records (921 calibration questions).
Nimble's train set is excluded (no license stated); the Jevals test items are excluded from training.
No private data of any kind.v1 is frozen. Later versions land as jebadiah-<size>-v<N> repositories. The adapter is Apache-2.0,
the base model is Apache-2.0. Evaluation data: Jevals suite 0.1.0 (CC-BY-4.0, "Jevals (jevals.com),
release 2026-09-18"), Nimble public subsets (Bespoke Labs), Kev test sets and typed-decisions test,
each under its own license.
| 1.0 / 1.1 |