myroslavtryhubets

clef-flash-NVFP4

README

License: apache-2.0

Usage

The quantized weights are not loaded by from_pretrained / load_release_model (those build the full-precision graph). Use the included clef_rt.py:

python

import sysfrom huggingface_hub import snapshot_download path = snapshot_download("myroslavtryhubets/clef-flash-NVFP4A16")sys.path.insert(0, path)from clef_rt import Clef model = Clef(path, "nvfp4-w4a16")                   # ≈4 GiB VRAM, Blackwell (sm_120) out = model.probs([model.encode({    "state": "Our checkout is down and orders are blocked.",    "questions": {        "outage": {"type": "noul", "instructions": "Is a service down?"},        "team": {"type": "choice", "instructions": "Which team should handle it?",                 "criteria": {"billing": "payments", "tech": "outages"}},    },})])[0]print(out)

Quality — Decision Index 0.2.1

11 benchmarks (20,337 requests) of the Decision Index 0.2.1 suite, rebuilt byte-identical with the official kit and scored with its own scorer. Clef-flash bf16 (CF) is Cloudflare's published result; This is this checkpoint on one RTX 5060 Ti 16 GB. Coverage-adjusted percentages.

Table with columns: Benchmark, Metric, Clef-flash bf16 (CF), This, Δ
Benchmark	Metric	Clef-flash bf16 (CF)	This	Δ
BFCL	case exact accuracy	98.8	98.6	-0.2
BANKING77	macro-F1	90.9	92.0	+1.1
CLINC150+OOS	macro-F1	66.8	61.2	-5.6
ContractNLI	macro-F1	84.3	83.7	-0.6
ANLI	macro-F1	59.1

Mean absolute gap to Cloudflare: 1.04 points. Median latency ≈1681 ms/request (batch 1).

Variants

Table with columns: Repo, Scheme, CLINC150, Mean gap, Median latency
Repo	Scheme	CLINC150	Mean gap	Median latency
clef-flash-NVFP4	W4A4	49.9	2.36	≈240 ms
clef-flash-NVFP4A16	W4 only	61.2	1.04	≈1680 ms
clef-NVFP4

Caveats

CLINC150 still -5.6 vs Cloudflare (66.8 -> 61.2) even weights-only: the 9B model is sensitive to weight quantization on 150 near-tied classes. Much better than W4A4 (49.9) but not lossless.
Slow for a "Flash" model (median ≈1.7s). On a 16 GB card Clef 27B NVFP4 is both faster (≈0.7s) and more accurate; pick this one only when 13 GB of VRAM for the 27B is not available. For speed use the W4A4 variant.
Quality verified only through this runtime (clef_rt.py), not a third-party serving stack.
vLLM: registers the Qwen3_5 backbone but has no Clef joint-head / decision path, so it cannot produce typed decisions out of the box.

Credits

Model, architecture and decision API by Cloudflare (blog). Base model Qwen/Qwen3.5-9B. Quantization by @myroslavtryhubets.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

myroslavtryhubets

Model Tree

Base

Cloudflare/clef-flash

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality