GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
EliovpAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
EliovpAI
Model Tree
Cloudflare/clef
Input Modalities
Output Modalities
Supported Functionality
| Format | OCP MXFP4: E2M1 elements with one E8M0 scale per 32 values along the input dimension |
| Weights | MXFP4, static, even scale rounding |
| Activations | MXFP4, dynamic per 32-value block |
| Algorithm | AWQ (Quark's qwen3_5 template) on the MLP projections |
| Calibration | 128 samples × 512 tokens from pileval (mit-han-lab/pile-val-backup) |
| Quantized | All language-model linear layers: full attention, Gated DeltaNet projections and MLP, in all 64 layers |
| Kept in BF16 | Vision encoder and merger, lm_head, embeddings, norms, the conv1d layers, and the joint schema head |
lm_head stays in BF16 deliberately. Clef's joint schema head reads the output-embedding matrix directly to embed answer
options.
The recipe is the same as AMD's own amd/Qwen3.8-27B-Quark-AWQ-MXFP4 for the base model:
bash
python3 quantize_quark.py \ --model_dir Cloudflare/clef \ --output_dir ./clef-MXFP4 \ --quant_scheme mxfp4 \ --quant_algo awq \ --num_calib_data 128 \ --seq_len 512 \ --model_export hf_format \ --data_type auto \ --device cuda \ --skip_evaluationquantize_quark.py is the LLM PTQ example from the Quark v0.13 repository.
After export, the backbone was resharded into five files with a safetensors index (bit-identical tensors), and the
joint head files were copied unchanged. algo_config is set to null in config.json because AWQ is already folded
into the weights and vLLM does not parse that field. The exported original is kept as config.json.orig_with_algo_config.
Use the original Clef code unchanged; only the repository name changes. Loading needs amd-quark:
bash
pip install amd-quark transformers pillowpython
import sys import torchfrom huggingface_hub import snapshot_download path = snapshot_download("EliovpAI/clef-MXFP4")sys.path.insert(0, path)from joint_schema_model import load_release_model, systemone model, processor = load_release_model(path, device="cuda") response = systemone(model, processor, { "model": "clef", "state": "Our checkout started returning errors and orders are blocked.", "questions": { "department": { "type": "choice", "instructions": "Which team should handle the message?", "criteria": {"billing": "Payments or invoices", "technical": "Bugs or outages"}, }, "urgency": {"type": "score", "criteria": ["Can wait", "This week", "Today"]}, "outage": {"type": "noul", "instructions": "Is a service down?"}, },})print(response["answers"])See the Clef model card for the input format, image and video inputs, and batching. In Transformers, Quark simulates the MXFP4 arithmetic. This is the reference path for accuracy, not for speed.
vLLM loads the quantized backbone and runs native MXFP4 GEMMs where the hardware supports them. The Clef joint head is not part of vLLM; use the Transformers path above for decisions.
bash
vllm serve EliovpAI/clef-MXFP4 --max-model-len 16384All runs were on one AMD Instinct MI355X (gfx950) with ROCm 7.2.3. BF16 is Cloudflare/clef at revision 2f3de3d.
Both models used the same code and inputs.
| Task | Records | BF16 accuracy | MXFP4 accuracy | Top-1 agreement | Mean KL (BF16 ‖ MXFP4) |
|---|---|---|---|---|---|
| BANKING77 test, 77-way choice | 300 | 94.0 % | 93.3 % | 98.0 % | 0.022 |
| CLINC150 plus test, 151-way choice incl. out-of-scope | 300 | 96.7 % | 96.7 % | 98.7 % | 0.018 |
| Receipt image, 2 × noul + 3-way choice |
The records are a fixed random sample (seed 0). Each label is a choice option whose description is the humanized label name. These are our own prompts, not the Decision Index protocol, so the absolute numbers are not comparable to Cloudflare's published results. Use the BF16/MXFP4 difference.
The SystemOne example from the Clef model card gives the same answers. technical is chosen at 0.918
(BF16 0.917), urgency peaks at "Today" with 0.920 (BF16 0.862), and outage is 0.841 (BF16 0.895).
| BF16 | MXFP4 | |
|---|---|---|
| WikiText-2 perplexity (64 × 1,024 tokens) | 9.974 | 10.505 (+5.3 %) |
| Greedy chat answers (4 prompts) | coherent | coherent, same content |
| File | Purpose |
|---|---|
model-0000X-of-00005.safetensors, model.safetensors.index.json | Quantized backbone (MXFP4 language model, BF16 vision encoder) |
config.json | Model and Quark quantization config |
config.json.orig_with_algo_config | Config as exported by Quark, including the AWQ settings |
joint_head.safetensors, joint_head_config.json, joint_schema_model.py | Clef joint schema head and code, unchanged from the original |
Apache-2.0, the same as Cloudflare/clef, which is post-trained from Qwen/Qwen3.8-27B. All credit for the model goes to Cloudflare and the Qwen team. This repository changes only the weight format of the backbone, as described above. It is not affiliated with or endorsed by Cloudflare, Qwen or AMD.
| 3 questions |
| – |
| – |
| 100 % |
| 0.0008 |
tokenizer*, chat_template.jinja, processor_config.json, preprocessor_config.json, generation_config.json | Tokenizer and processors |
LICENSE | Apache-2.0, from the original repository |
SHA256SUMS | Checksums of every file above |