GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
hotchpotch
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
hotchpotch
Model Tree
llm-jp/llm-jp-4.1-8b-thinking
Input Modalities
Output Modalities
Supported Functionality
| Repository | Description |
|---|---|
| saihai-v1-8b | This model (Transformers, BF16) |
| saihai-v1-8b-gguf | GGUF for llama.cpp's /v1/systemone endpoint (BF16 to Q2_K) |
| saihai-v1-310m | Small encoder model based on ruri-v3-reranker-310m |
| saihai-v1-dataset | Training data for this model |
Use Python 3.12+ with PyTorch and Transformers. The BF16 weights require
approximately 17.2 GB, plus inference memory. The bundled inference_saihai.py
runs standalone; installing the saihai package is unnecessary.
sh
uv pip install torch 'transformers==5.17.0' 'tokenizers>=0.23,<0.24' 'huggingface-hub>=1.0,<2' 'accelerate>=1.0,<2'The example asks whether a customer wants a refund (Noul), which department should handle the request (Choice), and how urgent it is (Score).
python
from transformers import AutoModel model = AutoModel.from_pretrained( "hotchpotch/saihai-v1-8b", trust_remote_code=True, device_map="auto", dtype="auto",)response = model.systemone({ 'state': '昨日購入した商品を返品したいです。返金をお願いします。', 'questions': { 'refund': {'type': 'noul', 'instructions': '顧客は返金を求めているか？'}, 'department': { 'type': 'choice', 'instructions': '問い合わせを担当部署に振り分ける。', 'criteria': { 'refunds': '返品・返金の対応', 'delivery': '配送状況・配達の対応', 'technical': 'ログイン・システム障害の対応', }, }, 'urgency': { 'type': 'score', 'instructions': '問い合わせの緊急度を評価する。', 'criteria': [ '通常対応：一般的な問い合わせや依頼', '優先対応：利用できず困っている', '至急対応：不正利用や被害が進行している', ], }, },})answers = response["answers"] print(f"返金要求あり：{answers['refund']['noul']:.2%}") department = answers["department"]department_names = { "refunds": "返品・返金", "delivery": "配送", "technical": "技術サポート",}selected = department["choice"]print( f"担当部署：{department_names[selected]}" f"（{department['probabilities'][selected]:.2%}）") urgency = answers["urgency"]probabilities = urgency["probabilities"]print( f"緊急度：通常対応 {probabilities['0']:.2%}、" f"優先対応 {probabilities['1']:.2%}、" f"至急対応 {probabilities['2']:.2%}" f"（スコア：{urgency['score']:.3f}）")Example output, verified with CPU / BF16 (small numerical differences may occur on other devices):
text
返金要求あり：97.84%担当部署：返品・返金（99.61%）緊急度：通常対応 82.53%、優先対応 15.27%、至急対応 2.20%（スコア：0.197）The urgency score is the probability-weighted average of normal 0, priority
1, and immediate 2. Probabilities are model outputs, not guaranteed
correctness rates.
Pin revision="<full commit hash>" for reproducibility. A single question can
use model.predict(state, question). For llama.cpp, use
saihai-v1-8b-gguf.
systemone() accepts state and named questions, and returns answers,
model and usage. Questions are evaluated independently.
noul is the probability of true; candidates are false then true.choice is the selected criterion key; probabilities contains
every candidate's probability. Supports 1–255 candidates.score is the expected zero-based rubric index. Supports 2–10
levels. Explicit numeric levels also return expected_value and numeric_values.Choice and Score also return confidence; probabilities are not separately
calibrated. Input JSON numbers use six significant digits; represent precise
quantities as strings.
The limit is 65,536 input tokens per question, including instructions, state, candidate definitions and model delimiters. Overflow is rejected without truncation. Training used an 8,192-token limit; quality on longer inputs, including 64K inputs, has not been established.
Results on S1MB (System One Mosaic Benchmark). Scores are baseline-adjusted 0–100 values, not raw accuracy. Task Avg is the equal-weight average of the Noul, Choice and Score means; "general" columns average only the synthetic generalization tasks.
| Benchmark | Task Avg | Noul | Choice | Score | General Noul | General Choice | General Score |
|---|---|---|---|---|---|---|---|
| japanese-v1 (27 tasks) | 49.77 | 48.40 | 61.78 | 39.11 | 52.22 | 69.31 | 21.01 |
| english-v1 (137 tasks) | 50.59 | 59.96 | 59.51 | 32.29 | 91.00 | 93.41 | 71.54 |
Compare with other decision models on the S1MB leaderboard. The Japanese evaluation covers 8,067 judgments with zero inference failures.
Training includes train-split examples from ten source datasets evaluated by S1MB japanese-v1 (for example JNLI, JSTS and JCommonsenseQA), and English source families from the Bekko System One dataset. Evaluation examples were excluded, but scores on those benchmarks are likely higher than on unseen tasks. These results do not establish generalization to unseen source families.
Trained for one epoch (2,096 optimizer steps) on all 33,532 train judgments from
hotchpotch/saihai-v1-dataset
at revision 906504ac4b72e133a613011318afd8206fb5e8ec: 17,509 Japanese and
16,023 English cases. Training used BF16 LoRA (rank 16, alpha 32), an FP32
decision head, effective batch size 16, an 8,192-token input limit and seed 42.
The final checkpoint was merged into BF16; validation did not select an earlier
checkpoint. Final validation soft-target cross entropy was 0.462753 across
279 fixed validation cases.
The decision head reads candidate-code logits in the same format as llama.cpp's
/v1/systemone endpoint (pplx-decider compatible), so the same weights also
run in llama.cpp after GGUF conversion.
See training-provenance.json for base/data revisions and training conditions,
and release-manifest.json for artifact hashes. The training, evaluation and
GGUF conversion code is at hotchpotch/saihai.
The model weights are licensed under Apache-2.0, the same as the base model. The training data has source-specific licenses; see the dataset card.
uv pip install torch 'transformers==5.17.0' 'tokenizers>=0.23,<0.24' 'huggingface-hub>=1.0,<2' 'accelerate>=1.0,<2'from transformers import AutoModel model = AutoModel.from_pretrained( "hotchpotch/saihai-v1-8b", trust_remote_code=True, device_map="auto", dtype="auto",)response = model.systemone({ 'state': '昨日購入した商品を返品したいです。返金をお願いします。', 'questions': { 'refund': {'type': 'noul', 'instructions': '顧客は返金を求めているか？'}, 'department': { 'type': 'choice', 'instructions': '問い合わせを担当部署に振り分ける。', 'criteria': { 'refunds': '返品・返金の対応', 'delivery': '配送状況・配達の対応', 'technical': 'ログイン・システム障害の対応', }, }, 'urgency': { 'type': 'score', 'instructions': '問い合わせの緊急度を評価する。', 'criteria': [ '通常対応：一般的な問い合わせや依頼', '優先対応：利用できず困っている', '至急対応：不正利用や被害が進行している', ], }, },})answers = response["answers"] print(f"返金要求あり：{answers['refund']['noul']:.2%}") department = answers["department"]department_names = { "refunds": "返品・返金", "delivery": "配送", "technical": "技術サポート",}selected = department["choice"]print( f"担当部署：{department_names[selected]}" f"（{department['probabilities'][selected]:.2%}）") urgency = answers["urgency"]probabilities = urgency["probabilities"]print( f"緊急度：通常対応 {probabilities['0']:.2%}、" f"優先対応 {probabilities['1']:.2%}、" f"至急対応 {probabilities['2']:.2%}" f"（スコア：{urgency['score']:.3f}）")返金要求あり：97.84%担当部署：返品・返金（99.61%）緊急度：通常対応 82.53%、優先対応 15.27%、至急対応 2.20%（スコア：0.197）