labqoat

qev-0.8b

README

License: apache-2.0

Which version

Table with columns: You have, Use, Latency (p50, text)
You have	Use	Latency (p50, text)
NVIDIA GPU (recommended)	this repo: `qev.py` / `qev_server.py` (fast CUDA engine)	3.5 ms (L40S)
Apple silicon	qev-0.8b-MLX	109 ms (M4 Pro)
llama.cpp (CPU, Metal, CUDA, Vulkan)	qev-0.8b-GGUF: `llama-server -hf labqoat/qev-0.8b-GGUF:Q8_0`	~17 ms (M4 Pro)
Anything else	this repo: `qev.py` (transformers backend)	not measured

Quickstart

pip install -r requirements.txt -r requirements-cuda.txt   # drop the second file without an NVIDIA GPU

python

from qev import QEV m = QEV.from_pretrained("labqoat/qev-0.8b")m.decide(    {"ticket": "My card was charged twice for the same order, please refund one."},    {"dept":   {"type": "choice", "instructions": "Which team should handle this?",                "criteria": {"billing": "payments and refunds", "tech": "bugs", "sales": None}},     "urgent": {"type": "noul", "instructions": "Is this urgent?"},     "anger":  {"type": "score", "instructions": "How upset is the customer?", "criteria": ["calm", "annoyed", "angry"]}})# {"dept": {"choice": "billing", "probabilities": {"billing": 0.95, ...}}, "urgent": {"noul": 0.16}, "anger": {"score": 1.05, ...}}

Server, same wire format as System One:

python qev_server.py   # http://127.0.0.1:8017/v1/systemonecurl http://127.0.0.1:8017/v1/systemone -H 'Content-Type: application/json' \  -d '{"state": {"ticket": "Refund please"}, "questions": {"urgent": {"type": "noul", "instructions": "Is this urgent?"}}}'

Images: put <image:NAME> in the state and pass images={"NAME": "shot.png"} (HTTP: "images": {name: base64}). Images are resized to at most 448×448 (≤196 tokens). Image support is zero-shot: the vision tower is the base model's.

Tips

Put the facts in the state and ask one short question per option.
Use per_question=True for image requests and for several related questions: it is more consistent and more accurate (JevBench coherence 64.5 → 71.2; JevVision 0.757 → 0.783) at some extra compute.
Act when it is confident; escalate when it is not.

Questions

Table with columns: type, asks, answer
type	asks	answer
`choice`	pick one named option (up to 255)	`choice`, `probabilities`, `confidence`
`noul`	yes / no	`noul` = P(yes)
`score`	a level on an ordered scale	`score` (expected level), `probabilities`

The prompt layout and answer format are decider-ai 1.9.0's. Probabilities are sharpened with per-type temperatures fitted on our own calibration split (choice 1.7, noul 0.9, score 1.25; in qev_config.json).

Benchmarks

Every model ran the same items and scoring on one L40S, in its own published runtime. Accuracy in %.

Table with columns: model, params, p50 ms, BH, BH ECE ↓, JevBench, suite, JevExtra, JevSpatial, Rules-7
model	params	p50 ms	BH	BH ECE ↓	JevBench	suite	JevExtra	JevSpatial	Rules-7
qev-0.8b	0.8B	3.5	77.1	0.063	85.1	61.7	72.0	74.2	63.7
decision-2b	2B	81.3	76.2	0.042	86.0	60.9	64.1	69.5	58.1
decider-2b	1.9B	7.2	75.8	0.065	85.9	70.7	58.8	–	–
EXAONE-4.0-1.2B-JEV	1.2B	27.8	68.8	0.097	83.9	67.3	64.0	70.0	58.8
malkuth-2b	2B	46.2	68.8	0.131	84.8	66.6	63.6	69.9	63.7

BH = Benchmark Heaven public (231 items, ±2.8 points). Suite = 22 public-dataset tasks. JevExtra, JevSpatial and Rules-7 are our own skill benchmarks (agents and safety; games and space; policy rules). Image input: 0.759 on our JevVision v0 benchmark, against 0.651 for the base model.

How to read these numbers. No benchmark item was used for training or for calibration, and the training data was screened against every benchmark item (no shared 16-token sequence with BH or JevBench). However:

benchmark scores were used to choose between checkpoints and weight soups, so they are not untouched held-out numbers;
part of the training data is written in the same formats as some benchmarks (for example long policy documents with amendments, sentiment scales, NLI, intent), because those are the formats the model is meant to serve;
JevExtra's indirect prompt-injection items share text with training rows, and its other skills share sources with training data.

Weak spots: 4-level score questions (JevBench score accuracy 0.39, mostly off by one), long or very hard inputs (BH long 0.51), choosing "cannot answer" / "no tool", and reading game boards or counting objects in images.

Training

Full fine-tune of Qwen3.5-0.8B-Base (dc7cdfe2) on about 2.4B tokens of decision data: a broad pass over 1.28M requests, anneals on newer data, a consistency pass (twin sets that must agree under rephrasing, reordering and complements), a weak-skill pass, and a games / rules / spatial pass. The release is a weight soup of two of these checkpoints.

Data: public datasets (about half the rows, with their own labels), program-generated data (about a third, exact labels), and rows written or labelled by larger models (the rest). Most rows are permissively licensed; about 10% are share-alike and under 1% research-use only. Each row records its source, license and how it was labelled. Whether those terms affect your use of the weights is for you to assess.

Limitations

score outputs are the weakest; treat them as soft signals.
Mostly English business, operations and agent decisions; other languages and domains are not evaluated.
Calibration was fitted on our data; recalibrate on your own labels when probabilities drive decisions.
It reads in one pass and does no reasoning; long, hard inputs are weaker.
Do not use it as the only decision maker for consequential decisions about people.

Files

model.safetensors (bf16 weights) · tokenizer and config files · qev_config.json (temperatures, option letters) · qev.py (runner) · qev_server.py (HTTP server) · qev_fast/ (CUDA engine) · qev_image.py, qev_image_server.py (transformers image path) · SHA256SUMS.

This release is the checkpoint called SG50 in our reports. License: Apache-2.0, as the base model.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

labqoat

Model Tree

Base

Qwen/Qwen3.5-0.8B-Base

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality