GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
szybkie-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
szybkie-ai
Model Tree
Qwen/Qwen3.5-4B
Input Modalities
Output Modalities
Supported Functionality
Accuracy on held-out public sets (validation/test splits, never trained on), raw probabilities (no post-hoc calibration); ECE in parentheses where available. Full tables, reliability diagrams and the Doom/Atari transfer results are in the technical report.
| set | n | base Qwen3.5-4B | JEB-4B | JEB-35B-A3B | Jev (hosted) |
|---|---|---|---|---|---|
| AG News | 300 | 0.857 (0.105) | 0.917 (0.043) | 0.913 (0.019) | 0.863 (0.095) |
| SST-2 | 300 | 0.933 (0.050) | 0.953 (0.021) | 0.960 (0.009) | 0.957 (0.102) |
| TREC | 300 | 0.773 (0.100) | 0.963 (0.021) | 0.973 (0.026) | 0.933 (0.025) |
| BoolQ | 300 | 0.847 (0.044) | 0.887 (0.031) | 0.920 (0.030) | 0.917 (0.022) |
| CLINC150 (20 options) | 400 | 0.960 (0.017) | 0.960 (0.038) | 0.980 (0.029) | 0.973 (0.012) |
| QNLI | 300 | 0.843 (0.053) | 0.883 (0.015) | 0.920 (0.037) | 0.930 (0.044) |
| RTE | 277 | 0.841 (0.064) | 0.892 (0.023) | 0.903 (0.022) | 0.913 (0.042) |
| IMDB | 300 | 0.947 (0.018) | 0.950 (0.033) | 0.957 (0.022) | 0.970 (0.049) |
| DBpedia | 300 | 0.987 (0.017) | 0.980 (0.036) | 0.973 (0.030) | 0.983 (0.015) |
| ARC-Easy | 300 | 0.980 (0.030) | 0.967 (0.023) | 0.987 (0.017) | 0.993 (0.007) |
| MMLU | 500 | 0.730 (0.052) | 0.706 (0.049) | 0.838 (0.034) | 0.920 (0.026) |
| blackjack (basic strategy) | 300 | 0.577 (0.050) | 0.543 (0.055) | 0.713 (0.096) | 0.837 (0.125) |
| tic-tac-toe (minimax) | 300 | 0.390 (0.067) | 0.393 (0.071) | 0.383 (0.054) | 0.427 (0.065) |
| document retrieval | 300 | 0.897 (0.023) | 0.947 (0.018) | 0.953 (0.014) | 0.910 (0.046) |
| claim vs passage | 172 | 0.866 (0.065) | 0.988 (0.015) | 0.988 (0.017) | 0.977 (0.028) |
| document MCQ (Polish) | 129 | 1.000 (0.018) | 0.992 (0.011) | 1.000 (0.007) | 1.000 (0.003) |
| document MCQ (Polish, medical) | 33 | 0.970 (0.028) | 0.970 (0.024) | 1.000 (0.019) | 1.000 (0.001) |
| document MCQ (English, medical) | 8 | 1.000 (0.033) | 1.000 (0.012) | 1.000 (0.006) | 1.000 (0.000) |
| MedQA (USMLE) | 300 | 0.683 (0.067) | 0.643 (0.092) | 0.897 (0.071) | 0.843 (0.050) |
| differential (6-8 candidates + none) | 300 | 0.480 (0.257) | 0.460 (0.093) | 0.637 (0.058) | 0.537 (0.213) |
| all 5,419 items | 0.804 (0.038) | 0.825 (0.007) | 0.878 (0.008) | 0.880 (0.014) |
bash
pip install git+https://github.com/szybkie-ai/jebvllm serve szybkie-ai/jeb-4b --served-model-name jeb-base --quantization fp8 --max-logprobs 32 --port 8021jeb serve --engine vllm-http --engine-url http://127.0.0.1:8021 --engine-model jeb-base --tokenizer szybkie-ai/jeb-4bcurl localhost:8020/v1/systemone -H 'Content-Type: application/json' -d '{"state": "I was charged twice for order A-104.", "model": "jeb-latest", "questions": {"refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"}}}'Or docker compose up in the JEB repository (vLLM + server).
JEB is a research model, published so that the method, the weights and the numbers can be examined and built upon. It is not a product and comes with no warranty or support. Intended use: classification, routing, extraction of typed judgements, and any decision inside software where a probability is more useful than a sentence; the output is a distribution over the options you give it, and it does not generate text.
Probabilities are calibrated on the evaluation mix above; on other distributions, measure calibration on your own data and set your own thresholds before relying on it. The medical rows in the training and evaluation mix exist to study calibration on hard multi-option decisions; the model must not be used for medical, legal, financial or safety decisions without independent validation and human review. Knowledge is that of the base model: the fine-tune changes how the model decides and how honest its probabilities are, not what it knows.
Base Qwen/Qwen3.5-4B; LoRA on all linear projections of all 32 layers; one epoch of ~42.5k rows; teacher-labelled soft targets ("mix": half gold one-hot,
half teacher distribution); cosine schedule. Data: public classification and entailment sets, knowledge MCQ
(ARC, OpenBookQA, SciQ, CommonsenseQA, MMLU auxiliary train); document question answering over Polish and English passages (multiple choice with gold, claim verification, passage retrieval); medical exam-style questions (MedQA) and differential-diagnosis cases built from published case reports, used for research on calibration only.
License: Apache-2.0 (weights), MIT (code). Made by szybkie.ai.
pip install git+https://github.com/szybkie-ai/jebvllm serve szybkie-ai/jeb-4b --served-model-name jeb-base --quantization fp8 --max-logprobs 32 --port 8021jeb serve --engine vllm-http --engine-url http://127.0.0.1:8021 --engine-model jeb-base --tokenizer szybkie-ai/jeb-4bcurl localhost:8020/v1/systemone -H 'Content-Type: application/json' -d '{"state": "I was charged twice for order A-104.", "model": "jeb-latest", "questions": {"refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"}}}'