GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
alexxorm
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
alexxorm
Model Tree
Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16
Input Modalities
Output Modalities
Supported Functionality
| Path | Size | What it is |
|---|---|---|
model-000NN-of-00018-int8.safetensors, model.safetensors.index.json | 33.3 GB | Backbone: int8 GPTQ linear layers + BF16 everything else (vision tower, MTP head included) |
config.json, generation_config.json | Model config with the GPTQ quantization_config (see Quantization) | |
tokenizer.json, tokenizer_config.json, vocab.json, merges.txt, chat_template.jinja | Tokenizer and chat template, unchanged from the base | |
preprocessor_config.json, video_preprocessor_config.json | Image / video processor, unchanged | |
clef/clef_lora_r256_fp16.safetensors | 1.32 GB | Exact rank-256 factors of W_clef − W_qwen, decoder layers 40–63 |
clef/deabl_abl_fp16.safetensors | 4.2 MB | Rank-1 factors of W_qwen − W_blackfrost (o_proj, out_proj, down_proj, layers 2–63) |
clef/joint_head_fp16.safetensors, clef/joint_head_config.json | 256 MB | Clef's joint schema head (128.1 M parameters), converted to fp16 |
LICENSE, NOTICE | Apache-2.0 and the attribution / modification notice | |
SHA256SUMS | Checksums of every file in this repository |
| Component | Source | Revision | License |
|---|---|---|---|
| Base model | Qwen/Qwen3.8-27B | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 | Apache-2.0 |
| De-risked BF16 master (quantized here) | Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16 | 9d85770e5eb602322b4bceef55beda357e0bd0ca | Apache-2.0 |
| Decision model (adapters derived from it) |
Related: Cloudflare/clef-flash, the smaller Clef variant (not used here).
qweight, qzeros stored as zero−1, scales, g_idx,
desc_act=false), so the checkpoint loads through vLLM's standard gptq method. At 8 bits, Hessian-based
GPTQ error compensation buys very little over RTN; the packing was validated against vLLM's
gptq_gemm (relative L2 1.09e-3 on a test layer). The script is
sm70/integrations/vllm_sm70/quantize/to_int8_gptq.py.mlp.{gate,up,down}_proj, self_attn.o_proj, linear_attn.{in_proj_qkv,in_proj_z,out_proj}
of decoder layers 1–63.Clef is post-trained from Qwen/Qwen3.8-27B. A tensor-by-tensor
comparison of the two checkpoints shows that Clef changes only the linear projections of decoder
layers 40–63 (all MLP, q/k/v/o, and all five Gated DeltaNet projections). Every difference has
rank exactly 256. Layers 0–39, norms, A_log/dt_bias/conv1d, embeddings, lm_head and the
whole vision tower are bit-identical to Qwen. The Blackfrost master differs from Qwen by a rank-1 edit
of o_proj, out_proj and down_proj in layers 2–63. Hence, for any linear layer:
text
W_clef = W_blackfrost + (W_qwen − W_blackfrost) + (W_clef − W_qwen) = W_blackfrost + deabl (rank 1) + LoRA (rank 256)Both corrections are stored exactly, as low-rank factors, in clef/. Before int8 quantization, the
reconstruction from the BF16 master plus the adapters matched Clef's own outputs to a median of
1.6e-3, which is the BF16 rounding floor. Without the adapters the error is 7.6e-2.
At serve time, vllm-sm70 applies the correction only to the rows of Clef requests inside the existing
int8 GEMM kernels: the cost is extra low-rank GEMMs on those rows plus streaming layers 40–63 of the LoRA
from pinned host memory. Base requests in the same batch are not modified. The final hidden states of a
Clef request are tapped into Clef's joint schema head. A native CUDA implementation reproduces the
reference PyTorch head to max |Δp| 1.5e-4. Clef requests carry their own cache_salt and skip prefix-cache reads,
so neither their KV cache nor their linear-attention state can leak into chat requests, and vice versa.
GPU memory cost on 2× V100: 290 MiB on rank 0 (head + buffers) and 121 MiB on rank 1.
Requires the vllm-sm70 stack. It is tested on 2× NVIDIA V100 32 GB (NVLink and PCIe), CUDA 12.8 (driver 580), Python 3.12.
bash
git clone -b sm70 https://github.com/xormal/vllm-sm70 && cd vllm-sm70scripts/install_from_source.sh # pinned venv + vLLM/flash-attention-v100 for sm_70huggingface-cli download alexxorm/Qwen3.8-27B-ABLITERATED-GPTQ-Int8-Clef --local-dir /models/Qwen3.8-27B-ABLITERATED-GPTQ-Int8-ClefPY=.venv/bin/python MODEL_DIR=/models/Qwen3.8-27B-ABLITERATED-GPTQ-Int8-Clef CARDS=0,1 PORT=8000 \ scripts/serve_qwen38_clef.shThe first start JIT-compiles the Volta kernels (≈15–30 min, cached afterwards). The Clef adapters are
picked up from MODEL_DIR/clef. Set FA2SM70_CLEF=0 to serve the base model only.
Chat (OpenAI-compatible):
bash
curl http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{ "model": "Qwen3.8-27B-ABLITERATED", "messages": [{"role": "user", "content": "Summarize the Volta memory hierarchy in two sentences."}]}'Decisions (Clef, SystemOne format):
bash
curl http://localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{ "model": "clef", "state": "Our checkout started returning errors and orders are blocked.", "questions": { "department": {"type": "choice", "instructions": "Which team should handle the message?", "criteria": {"billing": "Payments or invoices", "technical": "Bugs or outages"}}, "urgency": {"type": "score", "criteria": ["Can wait", "This week", "Today"]}, "outage": {"type": "noul", "instructions": "Is a service down?"}}}'json
{"model": "clef", "answers": { "department": {"type": "choice", "choice": "technical", "confidence": 0.9185, "probabilities": {"billing": 0.0815, "technical": 0.9185}}, "urgency": {"type": "score", "score": 1.8064, "confidence": 0.8546, "legend": {"0": "Can wait", "1": "This week", "2": "Today"}, "probabilities": {"0": 0.0482, "1": 0.0972, "2": 0.8546}}, "outage": {"type": "noul", "noul": 0.8999}}, "usage": {"input_tokens": 300, "output_tokens": 0}}Question types are choice (a dict of options), score (an ordered list), and noul (probability of
"yes"). state may be a string or JSON. Limits per record: 16,384 tokens (Clef's own default
max_length, which can be lowered per request with "max_length"), 64 questions, and 448 options in total.
Two records are processed concurrently and further ones queue; chat traffic proceeds in parallel.
All figures below are from the deployment of this exact checkpoint on 2× V100 32 GB. Stand runs used PCIe-connected cards; production runs used an NVLink pair.
| Check | Result |
|---|---|
| Clef decisions vs Cloudflare/clef in fp16 (10 records, 152–16,384 tokens, 35 decisions) | 35/35, max |Δp| 0.0093 (stand); 34/35 in production, where the one flip is a 0.200/0.199 near-tie in a deliberately meaningless record, which the reference itself scores 0.1916/0.1913 |
| Same records without the adapters (control) | 24/35, max |Δp| 0.64: the correction is necessary and effective |
| Clef determinism | bitwise-identical answers across repeats and server restarts |
| Mixed load: 10 Clef records + 6 chats concurrently, ×3 | 16/16 each round, no memory failures |
| Heavy order: 3 storms of 8 requests × 2 images (1280×960 + 640×480) with Clef records inside, then a 243,173-token needle, then images again | all answered; needle found; ≥ 782 MiB free at peak |
| Context capacity | 262,144 tokens (max_model_len) with int8 KV cache on two 32 GB cards |
Earlier measurements of the base model on the same stack (NVLink pair, MTP speculation with k=3): 250K-token prefill in 186.8 s (2026-09-15) and a median of 215 s (2026-09-23). Single-stream decode measured 55.8 / 58.4 / 48.6 tok/s at 32K / 128K / 250K context on filler text and 63.8 tok/s on natural text (2026-09-15). Prefill time depends on the cards' thermal state: V100s in hot slots throttle to about 1.3 GHz.
/v1/systemone the adapters undo the Blackfrost edit and
apply Clef's post-training, so decisions follow Cloudflare/clef up to int8 rounding (see Validation).
They are probabilities, not guarantees. Validate them for your schema before automating actions.encode_record. The native head
is verified up to 32,768 tokens, but Clef's behavior beyond 16K was not evaluated.sm_70). The base checkpoint is a standard GPTQ int8 checkpoint, but loading it on other stacks or GPUs was
not tested.Apache License 2.0, the same as all upstream components: Qwen3.8-27B, the Blackfrost derivative and
Cloudflare Clef. NOTICE lists the upstream works and the modifications made here (int8 quantization,
derivation of the low-rank adapters, fp16 conversion of Clef's head).
If you use this checkpoint, please cite the upstream works. Qwen3.8, as requested by the Qwen team:
bibtex
@misc{qwen38, title = {{Qwen3.8-Max}: A New Bar for Coding and Cowork}, url = {https://qwen.ai/blog?id=qwen3.8}, author = {{Qwen Team}}, month = {August}, year = {2026}}Clef: the Cloudflare blog post and the model card. The Blackfrost master: Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16.
The Qwen team for Qwen3.8-27B; Blackfrost for the BF16 master; Cloudflare for Clef and the SystemOne API; 1Cat AI for the Volta fork of vLLM; the vLLM, CUTLASS and LMDeploy/TurboMind projects.
| Cloudflare/clef (blog post, Decision Index) |
0b331204bb13fbd2ca93a64df1956f5b55478ce5 |
| Apache-2.0 |
| Serving stack | vllm-sm70, a fork of 1CatAI/1Cat-vLLM, itself a fork of vllm-project/vllm | sm70 branch | Apache-2.0 (+ MIT for the sm70/ kernels) |
| Kernel libraries used by the stack | NVIDIA/cutlass, InternLM/lmdeploy (TurboMind GEMM) | vendored | BSD-3-Clause, Apache-2.0 |
modules_to_not_convert): the vision tower, self_attn.{q,k,v}_proj, linear_attn.in_proj_{a,b},
the whole decoder layer 0, the MTP head, lm_head, embeddings and norms. The vllm-sm70 runtime
repacks q/k/v_proj to its own 12-bit format (W12) at load time and caches the result.config.json stays bfloat16. The serving stack
runs the model with --dtype float16.| Long-context retrieval | code word at the start of a 243,173-token prompt reproduced verbatim |