szybkie-ai

jeb-35b-a3b

README

License: apache-2.0

Numbers (round 3, 2026-09-20)

Accuracy on held-out public sets (validation/test splits, never trained on), raw probabilities (no post-hoc calibration); ECE in parentheses where available. Full tables, reliability diagrams and the Doom/Atari transfer results are in the technical report.

Table with columns: set, n, base Qwen3.6-35B-A3B, JEB-35B-A3B (round 3)
set	n	base Qwen3.6-35B-A3B	JEB-35B-A3B (round 3)
AG News	300	0.877 (0.091)	0.913 (0.019)
SST-2	300	0.957 (0.024)	0.960 (0.009)
TREC	300	0.937 (0.034)	0.973 (0.026)
BoolQ	300	0.880 (0.047)	0.920 (0.030)
CLINC150 (20 options)	400	0.968 (0.009)	0.980 (0.029)
QNLI	300	0.933 (0.036)	0.920 (0.037)
RTE	277	0.874 (0.040)	0.903 (0.022)
IMDB	300	0.960 (0.023)	0.957 (0.022)
DBpedia	300	0.983 (0.011)	0.973 (0.030)
ARC-Easy	300	0.990 (0.012)	0.987 (0.017)
MMLU (500)	500	0.818 (0.059)	0.838 (0.034)
STS-B	300	0.470 (0.168)	0.590 (0.075)
blackjack (basic strategy)	300	0.713 (0.171)	0.713 (0.096)
tic-tac-toe (minimax)	300	0.367 (0.188)	0.383 (0.054)
document retrieval (which passage / answerable)	300	0.930 (0.053)	0.953 (0.014)
claim supported by passage	172	0.983 (0.032)	0.988 (0.017)
MedQA (USMLE) test	300	0.880 (0.022)	0.897 (0.071)
differential over candidate conditions (+ 'none listed')	300	0.520 (0.304)	0.637 (0.058)
all 5,549 items		0.834 (0.062)	0.859 (0.009)

Run it

bash

pip install git+https://github.com/szybkie-ai/jebvllm serve szybkie-ai/jeb-35b-a3b --served-model-name jeb-base --quantization fp8 --max-logprobs 32 --port 8021jeb serve --engine vllm-http --engine-url http://127.0.0.1:8021 --engine-model jeb-base --tokenizer szybkie-ai/jeb-35b-a3bcurl localhost:8020/v1/systemone -H 'Content-Type: application/json' -d '{"state": "I was charged twice for order A-104.",  "model": "jeb-latest", "questions": {"refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"}}}'

Or docker compose up in the JEB repository (vLLM + server).

Intended use and limitations

JEB is a research model, published so that the method, the weights and the numbers can be examined and built upon. It is not a product and comes with no warranty or support. Intended use: classification, routing, extraction of typed judgements, and any decision inside software where a probability is more useful than a sentence; the output is a distribution over the options you give it, and it does not generate text.

Probabilities are calibrated on the evaluation mix above; on other distributions, measure calibration on your own data and set your own thresholds before relying on it. The medical rows in the training and evaluation mix exist to study calibration on hard multi-option decisions; the model must not be used for medical, legal, financial or safety decisions without independent validation and human review. Knowledge is that of the base model: the fine-tune changes how the model decides and how honest its probabilities are, not what it knows.

Training

Base Qwen/Qwen3.6-35B-A3B; LoRA on the attention and gated-delta projections and, per expert, on the expert weights of every MoE layer; one epoch of ~42.5k rows; teacher-labelled soft targets ("mix": half gold one-hot, half teacher distribution); cosine schedule. Data: public classification and entailment sets, knowledge MCQ (ARC, OpenBookQA, SciQ, CommonsenseQA, MMLU auxiliary train); document question answering over Polish and English passages (multiple choice with gold, claim verification, passage retrieval); medical exam-style questions (MedQA) and differential-diagnosis cases built from published case reports, used for research on calibration only.

License: Apache-2.0 (weights), MIT (code). Made by szybkie.ai.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

szybkie-ai

Model Tree

Base

Qwen/Qwen3.6-35B-A3B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality