aaroncool9

lev

README

License: apache-2.0

Installation

bash

pip install "lev[serve] @ git+https://github.com/Abhinavexists/lev#subdirectory=packages/lev"

This needs Python 3.12 or newer and, for real-time use, a CUDA GPU. The serve extra installs torch, transformers, peft, and the HTTP server. The first load downloads the base model (Qwen/Qwen3.5-4B, about 8 GB) and this adapter (about 200 MB).

Quickstart

python

import lev model = lev.load("interfaze-ai/lev") state = "Hi, I was charged twice for my order #4471 and I want a refund."questions = {    "intent": {        "type": "choice",        "instructions": "What does the customer want?",        "criteria": {            "refund": "wants money back",            "cancel": "wants to cancel an order",            "track": "wants to know where an order is",            "other": "anything else",        },    },    "urgent": {"type": "noul", "instructions": "Does this need a human within the hour?"},    "frustration": {        "type": "score",        "instructions": "How frustrated is the customer?",        "criteria": ["calm", "mildly annoyed", "annoyed", "angry"],    },} result = model.system_one(state, questions)print(result.answers["intent"].choice)  # refundprint(result.answers["intent"].probabilities)# {'refund': 0.84, 'cancel': 0.094, 'track': 0.012, 'other': 0.054}print(result.answers["urgent"].noul)  # 0.43print(result.answers["frustration"].score)  # 1.57, between "mildly annoyed" and "annoyed"print(result.usage.output_tokens)  # 0

These are real outputs from this checkpoint. The answers are objects, and result.model_dump() gives the same JSON the HTTP server returns.

All the questions share one forward pass, so asking three questions costs about the same as asking one. lev.load reads lev_release.json from this repository to find the base model and the prompt format the adapter was trained with. It also applies the shipped calibration and loads the matching head, so there is nothing to configure.

Because the probabilities are calibrated, you can gate on them. For example, act automatically above 0.9 and send anything lower to a person.

Self-hosting: a Jev-compatible HTTP server

bash

lev serve --checkpoint interfaze-ai/lev --host 0.0.0.0 --port 8000

bash

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{  "state": "The package arrived crushed and the screen is cracked.",  "questions": {"damaged": {"type": "noul", "instructions": "Was the item damaged?"}}}'

Existing TypeSafe clients work once you point them at the server:

python

from typesafe_sdk import Choice, Noul, TypeSafeClient # The first request after startup compiles kernels; allow more than the default 10 s.client = TypeSafeClient(base_url="http://localhost:8000", api_key="local", timeout=60)response = client.system_one(    state={"ticket": "The app crashes every time I open settings."},    questions={        "team": Choice(            instructions="Which team owns this?",            criteria={"billing": "payments, refunds", "technical": "bugs, crashes", "other": None},        ),        "bug": Noul(instructions="Is this a bug report?"),    },)print(response.answers["team"].choice, response.answers["bug"].noul)  # technical 0.92

GET /health reports the loaded checkpoint, whether calibration is active, and the routing settings. The server batches every question in a request into one forward pass, accepts concurrent requests, and returns 422 with the reason for a malformed question.

Hosted classification

Text classification in Interfaze runs on a similar system to lev: the model reads the answer from the set of labels you define. The difference is that Interfaze is still token based, so it's a hybrid.

Table with columns: Item, lev, Interfaze
Item	lev	Interfaze
Output	Probabilities, zero output tokens	Tokens, returned as structured output
Questions	Typed only: yes/no, choice, score	Any JSON schema, labels included
In the same request	Classification only	OCR, web search, transcription, extraction, and more
Where it runs	Your GPU	Interfaze API

bash

  pip install interfaze

python

  import os  from typing import Literal  from pydantic import BaseModel, Field  from interfaze import Interfaze    interfaze = Interfaze(api_key=os.environ["INTERFAZE_API_KEY"])    class Ticket(BaseModel):      intent: Literal["refund", "cancel", "track", "other"]      urgent: bool = Field(..., description="Does this need a human within the hour?")    response = interfaze.chat.completions.parse(      messages=[{"role": "user", "content": "Hi, I was charged twice for my order #4471 and I want a refund."}],      response_format=Ticket,  )    print(response.choices[0].message.parsed)

Tokens cost a little speed, but they let one request classify a document while also reading, searching, and extracting from it. Define your labels as an enum in the schema, and the label comes back as a typed field.

Interfaze docs → interfaze.ai/docs

Benchmarks

S1Bench

Table with columns: subset, task, lev, Jev, always the most common label
subset	task	lev	Jev	always the most common label
vitaminc-dev	claim verification	0.668	0.801	0.503
massive-en-US	intent routing, 18 scenarios	0.857	0.874	0.163
massive-de-DE	intent routing, German	0.823	0.871

lev and TypeSafe Jev ran through the same harness on all 3,880 items S1Bench scores, pinned by Nimble's manifests. Our Jev run lands within 0.8 points of TypeSafe's published figure on every subset, so the harness is not the gap.

Noise: at these sizes a per-subset difference needs roughly 5–9 points to be real. lev's leads on multinli and helpsteer2 are inside that.
Where Jev is clearly ahead: the minimal-edit pairs (paws −12.4, vitaminc −13.3) and summeval-consistency (−54.1), where lev rates most fully faithful summaries one level low. Fine-tuning introduced that: the untuned backbone scores 0.826.
Constant baselines: on civil_comments (89% not toxic) and the three 5-level rating subsets, always answering the most common label beats both models.
Calibration: mean ECE 0.115 for lev, 0.091 for Jev; lev is better calibrated on 5 of 13.

Held-out split

A held-out split of the 29 training sources, with no row shared with training:

Table with columns: metric, value
metric	value
weighted accuracy	0.807*
expected calibration error	0.061* (0.180 before calibration)
banking77 (77 intents)	0.980
clinc_oos (151 intents)	0.968
FEVER claim verification	0.872*

* Measured on this checkpoint before the last serving update. That update routes choice sets of more than 68 options to label-token readout and re-selects the temperatures. The banking77 and clinc_oos rows come from after the update, which raised banking77 from 0.818. Smaller option sets are routed the same way as before.

Speed

A call is one batched forward pass over every question, so compute stays flat from one question to eight, and a 60-option choice costs the same as a yes/no.

The 69 ms is engine compute for a short request (a three-sentence state), measured inside the container; S1Bench's longer states take more. End to end from a laptop, Jev's hosted API answered in 335–346 ms median and lev on one Modal H100 in 414–654 ms across two runs.

Why it works

Label-token readout. Each option gets a short code, and the answer is read from the next-token logits over those codes. Codes that would split into two tokens are skipped, so every option stays one token. This handles yes/no, scores, and choice lists up to several hundred options. Choices are read in two option orders and averaged, which cancels position bias.
Candidate-path head. Past that point, a small learned head matches the state against each option's text, so the number of options has no fixed ceiling.
Calibration. Temperatures fitted after training are applied at load time. There is one per question type, readout mode, and choice option-count band. They were selected by how well they carry over to task families left out of the fit, not only by how well they fit held-out rows.

The optimizations that mattered

One batched forward instead of prefill-and-fork: 169 → 69 ms. At this size the forward pass is bound by kernel launches, not arithmetic, so forking the cache saved FLOPs and cost time. One batched forward plus the depthwise-conv kernel cut compute by 59%. ADR-023
Label-token readout up to the tokenizer's limit: +51 points on MASSIVE's 60 intents. Serving 60 options through label codes instead of the learned head took accuracy from 0.231 to 0.746. ADR-025
Skipping codes that split: banking77 0.818 → 0.980. Passing over codes that tokenize to two tokens lifts label-token readout from 68 options to several hundred. ADR-028
The backbone's own prompt format: 0.653 → 0.710 frozen. The untuned instruct model scored 5.7 points higher on the earlier six-subset S1Bench set when questions are dressed in its chat template, so the final run trained in that format. ADR-027

Training

Data: 200,000 examples from 29 sources built on 26 public Hugging Face datasets. The tasks cover topic, sentiment, and emotion classification; intent detection (banking77, clinc_oos, snips); NLI (SNLI, ANLI, FEVER); paraphrase (MRPC, QQP, PARADE, plus word-swapped hard negatives); multiple-choice QA (RACE, ARC, SciQ, OpenBookQA, CommonsenseQA, StrategyQA); toxicity and safety (ToxiGen, ToxicChat, BeaverTails); and helpfulness (UltraFeedback). Questions are paraphrased, negated, and recast between types, and option sets are shuffled and resized, so the model learns the question format and not one wording.
Contamination guard: the data build refuses any source that resolves to one of the 13 S1Bench subsets.
Recipe: LoRA r=32, α=64 on the q/k/v/o attention and MLP projections; 3 epochs, 18,750 steps, batch size 32, learning rate 5e-5, in the base model's chat format, on one H100 in 7.8 hours.

Boundaries worth understanding

Minimal edits and fine-grained ratings are weak. Inputs that differ by one swapped word or number, and quality ratings over five levels, are where lev is least accurate and can be confidently wrong.
Calibration is fitted on the training distribution. Temperatures are chosen to transfer across task families. Even so, a task very unlike the training mix may be less well calibrated. Check on your own data before you gate on the probabilities.
Questions are answered independently. Answers in one request do not condition on each other. Encode a joint decision as one choice, or ask in stages.
English only.
Needs a GPU for real-time use. It runs on CPU, but a 4B backbone there takes seconds per call, not milliseconds.

Files

Table with columns: file, role
file	role
`adapter_model.safetensors`, `adapter_config.json`	LoRA adapter
`mode_b_head.pt`	candidate-path head (tensor state dict, loaded with `weights_only=True`)
`tokenizer*`, `chat_template.jinja`	the tokenizer that the label codes were verified against
`calibration.json`	fitted temperatures

License

The adapter is released under Apache-2.0, the same license as the base model. Some of the training datasets have their own terms, including non-commercial licenses. Review them before commercial use.

Not affiliated with or endorsed by TypeSafe AI.

Apache 2.0 · Interfaze

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

aaroncool9

Model Tree

Base

Qwen/Qwen3.5-4B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Installation

bash

pip install "lev[serve] @ git+https://github.com/Abhinavexists/lev#subdirectory=packages/lev"

Quickstart

python

import lev model = lev.load("interfaze-ai/lev") state = "Hi, I was charged twice for my order #4471 and I want a refund."questions = {    "intent": {        "type": "choice",        "instructions": "What does the customer want?",        "criteria": {            "refund": "wants money back",            "cancel": "wants to cancel an order",            "track": "wants to know where an order is",            "other": "anything else",        },    },    "urgent": {"type": "noul", "instructions": "Does this need a human within the hour?"},    "frustration": {        "type": "score",        "instructions": "How frustrated is the customer?",        "criteria": ["calm", "mildly annoyed", "annoyed", "angry"],    },} result = model.system_one(state, questions)print(result.answers["intent"].choice)  # refundprint(result.answers["intent"].probabilities)# {'refund': 0.84, 'cancel': 0.094, 'track': 0.012, 'other': 0.054}print(result.answers["urgent"].noul)  # 0.43print(result.answers["frustration"].score)  # 1.57, between "mildly annoyed" and "annoyed"print(result.usage.output_tokens)  # 0

These are real outputs from this checkpoint. The answers are objects, and result.model_dump() gives the same JSON the HTTP server returns.

Because the probabilities are calibrated, you can gate on them. For example, act automatically above 0.9 and send anything lower to a person.

Self-hosting: a Jev-compatible HTTP server

bash

lev serve --checkpoint interfaze-ai/lev --host 0.0.0.0 --port 8000

bash

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{  "state": "The package arrived crushed and the screen is cracked.",  "questions": {"damaged": {"type": "noul", "instructions": "Was the item damaged?"}}}'

Existing TypeSafe clients work once you point them at the server:

python

from typesafe_sdk import Choice, Noul, TypeSafeClient # The first request after startup compiles kernels; allow more than the default 10 s.client = TypeSafeClient(base_url="http://localhost:8000", api_key="local", timeout=60)response = client.system_one(    state={"ticket": "The app crashes every time I open settings."},    questions={        "team": Choice(            instructions="Which team owns this?",            criteria={"billing": "payments, refunds", "technical": "bugs, crashes", "other": None},        ),        "bug": Noul(instructions="Is this a bug report?"),    },)print(response.answers["team"].choice, response.answers["bug"].noul)  # technical 0.92

Hosted classification

Table with columns: Item, lev, Interfaze
Item	lev	Interfaze
Output	Probabilities, zero output tokens	Tokens, returned as structured output
Questions	Typed only: yes/no, choice, score	Any JSON schema, labels included
In the same request	Classification only	OCR, web search, transcription, extraction, and more
Where it runs	Your GPU	Interfaze API

bash

  pip install interfaze

python

  import os  from typing import Literal  from pydantic import BaseModel, Field  from interfaze import Interfaze    interfaze = Interfaze(api_key=os.environ["INTERFAZE_API_KEY"])    class Ticket(BaseModel):      intent: Literal["refund", "cancel", "track", "other"]      urgent: bool = Field(..., description="Does this need a human within the hour?")    response = interfaze.chat.completions.parse(      messages=[{"role": "user", "content": "Hi, I was charged twice for my order #4471 and I want a refund."}],      response_format=Ticket,  )    print(response.choices[0].message.parsed)

Interfaze docs → interfaze.ai/docs

Benchmarks

S1Bench

Table with columns: subset, task, lev, Jev, always the most common label
subset	task	lev	Jev	always the most common label
vitaminc-dev	claim verification	0.668	0.801	0.503
massive-en-US	intent routing, 18 scenarios	0.857	0.874	0.163
massive-de-DE	intent routing, German	0.823	0.871

Noise: at these sizes a per-subset difference needs roughly 5–9 points to be real. lev's leads on multinli and helpsteer2 are inside that.
Where Jev is clearly ahead: the minimal-edit pairs (paws −12.4, vitaminc −13.3) and summeval-consistency (−54.1), where lev rates most fully faithful summaries one level low. Fine-tuning introduced that: the untuned backbone scores 0.826.
Constant baselines: on civil_comments (89% not toxic) and the three 5-level rating subsets, always answering the most common label beats both models.
Calibration: mean ECE 0.115 for lev, 0.091 for Jev; lev is better calibrated on 5 of 13.

Held-out split

A held-out split of the 29 training sources, with no row shared with training:

Table with columns: metric, value
metric	value
weighted accuracy	0.807*
expected calibration error	0.061* (0.180 before calibration)
banking77 (77 intents)	0.980
clinc_oos (151 intents)	0.968
FEVER claim verification	0.872*

Speed

A call is one batched forward pass over every question, so compute stays flat from one question to eight, and a 60-option choice costs the same as a yes/no.

Why it works

Label-token readout. Each option gets a short code, and the answer is read from the next-token logits over those codes. Codes that would split into two tokens are skipped, so every option stays one token. This handles yes/no, scores, and choice lists up to several hundred options. Choices are read in two option orders and averaged, which cancels position bias.
Candidate-path head. Past that point, a small learned head matches the state against each option's text, so the number of options has no fixed ceiling.
Calibration. Temperatures fitted after training are applied at load time. There is one per question type, readout mode, and choice option-count band. They were selected by how well they carry over to task families left out of the fit, not only by how well they fit held-out rows.

The optimizations that mattered

One batched forward instead of prefill-and-fork: 169 → 69 ms. At this size the forward pass is bound by kernel launches, not arithmetic, so forking the cache saved FLOPs and cost time. One batched forward plus the depthwise-conv kernel cut compute by 59%. ADR-023
Label-token readout up to the tokenizer's limit: +51 points on MASSIVE's 60 intents. Serving 60 options through label codes instead of the learned head took accuracy from 0.231 to 0.746. ADR-025
Skipping codes that split: banking77 0.818 → 0.980. Passing over codes that tokenize to two tokens lifts label-token readout from 68 options to several hundred. ADR-028
The backbone's own prompt format: 0.653 → 0.710 frozen. The untuned instruct model scored 5.7 points higher on the earlier six-subset S1Bench set when questions are dressed in its chat template, so the final run trained in that format. ADR-027

Training

Data: 200,000 examples from 29 sources built on 26 public Hugging Face datasets. The tasks cover topic, sentiment, and emotion classification; intent detection (banking77, clinc_oos, snips); NLI (SNLI, ANLI, FEVER); paraphrase (MRPC, QQP, PARADE, plus word-swapped hard negatives); multiple-choice QA (RACE, ARC, SciQ, OpenBookQA, CommonsenseQA, StrategyQA); toxicity and safety (ToxiGen, ToxicChat, BeaverTails); and helpfulness (UltraFeedback). Questions are paraphrased, negated, and recast between types, and option sets are shuffled and resized, so the model learns the question format and not one wording.
Contamination guard: the data build refuses any source that resolves to one of the 13 S1Bench subsets.
Recipe: LoRA r=32, α=64 on the q/k/v/o attention and MLP projections; 3 epochs, 18,750 steps, batch size 32, learning rate 5e-5, in the base model's chat format, on one H100 in 7.8 hours.

Boundaries worth understanding

Minimal edits and fine-grained ratings are weak. Inputs that differ by one swapped word or number, and quality ratings over five levels, are where lev is least accurate and can be confidently wrong.
Calibration is fitted on the training distribution. Temperatures are chosen to transfer across task families. Even so, a task very unlike the training mix may be less well calibrated. Check on your own data before you gate on the probabilities.
Questions are answered independently. Answers in one request do not condition on each other. Encode a joint decision as one choice, or ask in stages.
English only.
Needs a GPU for real-time use. It runs on CPU, but a 4B backbone there takes seconds per call, not milliseconds.

Files

Table with columns: file, role
file	role
`adapter_model.safetensors`, `adapter_config.json`	LoRA adapter
`mode_b_head.pt`	candidate-path head (tensor state dict, loaded with `weights_only=True`)
`tokenizer*`, `chat_template.jinja`	the tokenizer that the label codes were verified against
`calibration.json`	fitted temperatures

License

The adapter is released under Apache-2.0, the same license as the base model. Some of the training datasets have their own terms, including non-commercial licenses. Review them before commercial use.

Not affiliated with or endorsed by TypeSafe AI.

Apache 2.0 · Interfaze