GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
j-raghavan
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
j-raghavan
Model Tree
google/gemma-4-26B-A4B-it
Input Modalities
Output Modalities
Supported Functionality
Held-out test sets. The released snapshot was chosen on separate dev splits by a rule fixed before training.
| System | JevBench subset (1,700 cases) raw / calibrated | Custom structured decisions raw / calibrated | Prompt injections caught / false alarms |
|---|---|---|---|
| s1 (this model, bf16) | 0.808 / 0.817 | 0.971 / 0.981 | 45/53, 0 |
| Gemma 4 26B-A4B, untuned | 0.770 / 0.799 | 0.948 / 0.960 | 35/53, 0 |
| s1, int4 through Ollama (Apple-silicon laptop) | 0.801 / 0.804 | 0.966 / 0.981 | 46/53, 0 |
| Jev 1.13 (closed; predictions published with JevBench) | 0.835 / 0.856 | — | — |
eval/jevbench/calibrate_from_dev.py). That
study covers the cases with usable option log-probabilities, where s1's raw JevBench score is 0.804 rather than 0.808.Latency per decision on one NVIDIA RTX PRO 6000 through vLLM: median 27 ms (p95 121 ms) with FP8 weights, 39 ms (155 ms) in bf16.
The prompt format matters. Render the question with lettered options, start the prompt with <bos> (Gemma 4's
tokenizer does not add it, and accuracy drops by about 3 points without it), prefill the reply with Answer:, and
read the next-token logits of " A", " B", ... Load the model with Gemma4ForConditionalGeneration; a plain
causal-LM class silently skips the language-model weights.
python
import jsonimport torchfrom transformers import AutoTokenizer, Gemma4ForConditionalGeneration MODEL = "j-raghavan/s1-gemma4-26b-decision"TEMPLATE = "<|turn>user\n{user}<turn|>\n<|turn>model\n<|channel>thought\n<channel|>"tok = AutoTokenizer.from_pretrained(MODEL)model = Gemma4ForConditionalGeneration.from_pretrained(MODEL, dtype=torch.bfloat16, device_map="auto").eval() def decide(state, instructions, options): # options: {key: description}, 2 to 26 of them keys = list(options) lines = "\n".join(f"{chr(65 + i)}) {k}: {options[k]}" for i, k in enumerate(keys)) user = ("You are a decision model. Read the state and answer the question by choosing one option.\n\n" f"STATE:\n{json.dumps(state, ensure_ascii=False, indent=1)}\n\nQUESTION: {instructions}\n\n" f"OPTIONS:\n{lines}\n\nAnswer with the option letter only.") enc = tok("<bos>" + TEMPLATE.format(user=user) + "Answer:", return_tensors="pt", add_special_tokens=False).to(model.device) with torch.no_grad(): logits = model(**enc, logits_to_keep=1).logits[0, -1] ids = [tok.encode(" " + chr(65 + i), add_special_tokens=False)[0] for i in range(len(keys))] return dict(zip(keys, torch.softmax(logits[ids].float(), -1).tolist())) print(decide({"ticket": "I was charged twice for order 4471."}, "Which team should handle this ticket?", {"billing": "Payments and refunds", "shipping": "Deliveries", "tech": "Bugs and outages"}))This short version writes every option as key: description. The complete version (yes/no and ordinal questions,
and option formatting identical to training, checked by tests) is
examples/quickstart.py. For a
calibrated HTTP endpoint (POST /v1/decisions), see api/server.py in the repository.
I served and measured s1 with vLLM 0.31.0. Run the model behind vLLM's OpenAI-compatible server and put the
repository's API in front of it; the API builds the prompt exactly as in training, prepends <bos>, reads the option
probabilities and applies calibration fitted on bf16 vLLM dev predictions (api/calibration_s1_vllm.json):
bash
pip install vllm==0.31.0vllm serve j-raghavan/s1-gemma4-26b-decision --port 8001 \ --max-model-len 8192 --max-logprobs 20 --gpu-memory-utilization 0.90 --enforce-eager# add --quantization fp8 for FP8 weights on cards with FP8 support uv sync --extra apiS1_BACKEND=vllm VLLM_URL=http://localhost:8001 uv run --extra api uvicorn api.server:app --port 8000curl -s localhost:8000/healthz # ok: true once vLLM serves the model and the calibration loads/healthz returns 503 until vLLM lists the model, so it works as a readiness probe. Set S1_CHOICE_ORDERS=2 to
score each choice question a second time with its options reversed and average the two. On the JevBench test set
this cut answers that change when options are reordered from 9.6% to 6.0% (Jev: 4.8%), with accuracy 0.814 vs
0.809 (not a significant difference) and ECE 0.081 vs 0.085, for about 75 ms instead of 38 ms per choice decision in
bf16. I chose it on the dev split and then scored the test set with it once; yes/no and ordinal questions keep one
pass, because reversing yes/no options cost accuracy on dev. If you start vLLM with
--served-model-name, set S1_MODEL to the same name. If you call vLLM directly instead, send /v1/completions (not
chat) with max_tokens: 1, temperature: 0 and logprobs: 20 on a prompt that starts with the literal <bos>;
without it the model loses 4 to 13 points on knowledge questions.
Sizing: I ran exactly these commands on the published weights, downloaded from the Hub, on one 96 GB RTX PRO 6000.
| bf16 | FP8 (--quantization fp8) | |
|---|---|---|
| Weights loaded (vLLM's count) | 48.5 GiB | 25.8 GiB |
KV cache left at --gpu-memory-utilization 0.90 | 34.4 GiB, about 20 requests at 8,192 tokens | 56.5 GiB, about 33 requests at 8,192 tokens |
| GPU | 80 GB or more (H100, H200, A100 80 GB, RTX PRO 6000) | a card with FP8 support (Ada, Hopper or Blackwell, such as L40S, H100) |
| JevBench test, raw / dev-calibrated | 0.811 / 0.817 | 0.809 / 0.816 |
| Latency per decision, vLLM, one request at a time | 39 ms median, 155 ms p95 | 27 ms median, 121 ms p95 |
| Latency through the API, one request at a time | 51 ms median, 208 ms p95 | 40 ms median, 248 ms p95 |
--enforce-eager, the configuration I measured; CUDA graphs (dropping the flag) are usually faster.curl from the API section should put most of the probability on
billing, and examples/quickstart.py against the same weights should give the same top answer.bash
ollama pull jrlabs01/s1 # the int4 build, about 17 GB on diskpython3 examples/ollama_decide.py --state '{"ticket": "I was charged twice."}' \ --question "Which team should handle this?" --option billing=Payments --option tech=Bugsexamples/ollama_decide.py (in the GitHub repository) builds the prompt and reads the probabilities correctly; Ollama
lists two spellings of each option letter (" A" and "A"), and only the first, most likely one must be kept.
For decisions, send raw prompts ("raw": true) that start with <bos> and end with the Answer: prefill: that is
exactly the training prompt. The repository's API does this for you
(S1_MODEL=jrlabs01/s1 uv run --extra api uvicorn api.server:app), with calibration fitted on the int4 build's own dev
predictions (api/calibration_s1.json).
The chat API (/api/chat, ollama run) works as well. I configured the build with Ollama's gemma4-large renderer
and thinking turned off (scripts/ollama_configure.py), so a chat message renders as the training prompt without the
Answer: prefill, and the model answers in one token instead of reasoning first. On the full JevBench test set sent
as chat messages, the option probabilities read from the reply's first-token logprobs score 0.801 (ECE 0.077), the
same as the raw path (0.801, ECE 0.074); paired against raw on the same machine, 23 answers are worse and 16 better
(p = 0.34). Reading the reply text instead scores 0.800, and 6 of 1,592 replies gave no usable letter, so use the
probabilities when you need them.
On Linux, this build needs Ollama's MLX runtime, which the standard install script leaves out; without it,
ollama pull stops with "this model requires MLX support". Install the add-on from the same release
(ollama-linux-amd64-mlx.tar.zst, extracted into Ollama's install prefix, /usr/local by default). On an NVIDIA GPU
it ran the full test set at 8 parallel requests in 13.6 minutes on the raw path, and its runner restarted twice
asking for MLX_CUDA_GRAPH_CACHE_SIZE above 400 (15 of 1,700 requests failed). For NVIDIA servers, vLLM (above) is
the faster and better-tested path.
To build it yourself instead: download this repo, write a Modelfile with FROM <folder>, run
ollama create <name> --quantize int4 -f Modelfile, then python3 scripts/ollama_configure.py --source <name> --target <name>.
Measured cost of int4 against bf16 on the test sets: JevBench 0.801 vs 0.808 (the same answer on 92% of cases; paired 36 better, 48 worse, p = 0.23) and custom 0.966 vs 0.971 (p = 0.18); neither difference is significant. On a 32 GB Apple-silicon laptop it takes about 0.8 s per decision (median); Ollama holds about 26 GB while it is loaded, so a 32 GB machine swaps under other load.
Hardware: about 52 GB in bf16 (one 80–96 GB GPU, or device_map="auto" across several GPUs). If the model does not
fit, device_map="auto" offloads layers and the forward pass fails unless you also pass offload_folder (slow); use a
quantised build instead.
openai/gpt-oss-120b (kept only where Gemma 4 independently agrees),
and prompt-injection sets (SPML, Lakera Gandalf, LLMail-Inject). Full card:
j-raghavan/s1-decision-data.Methodology and decision rules: docs/methodology.md.
Apache-2.0, like the base model. This model is a modified version of google/gemma-4-26B-A4B-it: fine-tuned with LoRA and merged. It is not endorsed by Google. Training data keep their source licences (Apache-2.0, MIT, BSD, CC0, CC-BY and CC-BY-SA); see the dataset card. s1 is an independent project, not affiliated with TypeSafe AI (Jev) or the JevBench authors.
bibtex
@software{raghavan2026s1, author = {Raghavan, Jayasimha}, title = {s1: an open, calibrated System One decision model}, year = {2026}, url = {https://github.com/j-raghavan/s1-decision-model}}import jsonimport torchfrom transformers import AutoTokenizer, Gemma4ForConditionalGeneration MODEL = "j-raghavan/s1-gemma4-26b-decision"TEMPLATE = "<|turn>user\n{user}<turn|>\n<|turn>model\n<|channel>thought\n<channel|>"tok = AutoTokenizer.from_pretrained(MODEL)model = Gemma4ForConditionalGeneration.from_pretrained(MODEL, dtype=torch.bfloat16, device_map="auto").eval() def decide(state, instructions, options): # options: {key: description}, 2 to 26 of them keys = list(options) lines = "\n".join(f"{chr(65 + i)}) {k}: {options[k]}" for i, k in enumerate(keys)) user = ("You are a decision model. Read the state and answer the question by choosing one option.\n\n" f"STATE:\n{json.dumps(state, ensure_ascii=False, indent=1)}\n\nQUESTION: {instructions}\n\n" f"OPTIONS:\n{lines}\n\nAnswer with the option letter only.") enc = tok("<bos>" + TEMPLATE.format(user=user) + "Answer:", return_tensors="pt", add_special_tokens=False).to(model.device) with torch.no_grad(): logits = model(**enc, logits_to_keep=1).logits[0, -1] ids = [tok.encode(" " + chr(65 + i), add_special_tokens=False)[0] for i in range(len(keys))] return dict(zip(keys, torch.softmax(logits[ids].float(), -1).tolist())) print(decide({"ticket": "I was charged twice for order 4471."}, "Which team should handle this ticket?", {"billing": "Payments and refunds", "shipping": "Deliveries", "tech": "Bugs and outages"}))pip install vllm==0.31.0vllm serve j-raghavan/s1-gemma4-26b-decision --port 8001 \ --max-model-len 8192 --max-logprobs 20 --gpu-memory-utilization 0.90 --enforce-eager# add --quantization fp8 for FP8 weights on cards with FP8 support uv sync --extra apiS1_BACKEND=vllm VLLM_URL=http://localhost:8001 uv run --extra api uvicorn api.server:app --port 8000curl -s localhost:8000/healthz # ok: true once vLLM serves the model and the calibration loadsollama pull jrlabs01/s1 # the int4 build, about 17 GB on diskpython3 examples/ollama_decide.py --state '{"ticket": "I was charged twice."}' \ --question "Which team should handle this?" --option billing=Payments --option tech=Bugs@software{raghavan2026s1, author = {Raghavan, Jayasimha}, title = {s1: an open, calibrated System One decision model}, year = {2026}, url = {https://github.com/j-raghavan/s1-decision-model}}