GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
norecyc
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
norecyc
Model Tree
speakleash/Bielik-11B-v3.0-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Base model | speakleash/Bielik-11B-v3.0-Instruct (Llama arch, 11B) |
| Method | LLM Compressor one-shot |
| Scheme | NVFP4A16 — weight-only (4-bit NVFP4 weights, BF16 activations) |
| Format | compressed-tensors (nvfp4-pack-quantized), group size 16, FP8-E4M3 block scales |
| Not quantized | lm_head |
| Size | ~6.3 GB (from ~21 GB BF16) |
| License | Apache-2.0 (inherited from base) |
NVFP4 runs natively on NVIDIA Blackwell (GB10 / DGX Spark, B200, RTX 50-series). Built and tested on NVIDIA GB10 (Grace Blackwell).
bash
vllm serve norecyc/Bielik-11B-v3.0-Instruct-NVFP4 \ --served-model-name bielik-11b \ --speculative-config '{"method":"dflash","model":"speakleash/Bielik-11B-v3.0-DFlash","num_speculative_tokens":8}' \ --max-model-len 32768 --gpu-memory-utilization 0.85 \ --host 0.0.0.0 --port 8000bash
vllm serve norecyc/Bielik-11B-v3.0-Instruct-NVFP4 --max-model-len 32768Use the model's native chat template (bundled chat_template.jinja, ChatML-style). Do not use a tool-calling template (e.g. Hermes) for plain chat — it makes the model emit tool calls instead of answering.
Quality vs the BF16 base, measured with lm-evaluation-harness (loglikelihood / multiple-choice — speculative decoding does not affect these scores, so this isolates quantization quality), n=200 per task:
| Task | BF16 base | NVFP4 (this model) | Delta |
|---|---|---|---|
| arc_challenge (reasoning) | 0.555 | 0.575 | +0.020 |
| belebele_pol (Polish reading) | 0.845 | 0.855 | +0.010 |
Both differences are within statistical noise (~±3% at n=200) — no measurable quality regression from 4-bit quantization. (n=200 rules out degradation beyond ~3-4%; for sub-1% certainty, run full task sets.)
Single-stream decode throughput on one NVIDIA GB10 (temp=0), decomposing the two speedup levers — NVFP4 quantization x DFlash speculative decoding:
| DFlash off | DFlash on | |
|---|---|---|
| BF16 | 9.6 tok/s | 25.9 tok/s |
| NVFP4 | 33.7 tok/s | 73-78 tok/s |
num_speculative_tokens = 8; speculative mean acceptance length ~3.2 of 8.Quantizing the DFlash draft to NVFP4 is not currently loadable in vLLM (the dedicated
qwen3_dflashdraft loader does not support compressed-tensors weight packing), so the draft is served in BF16.
python
from llmcompressor import oneshotfrom llmcompressor.modifiers.quantization import QuantizationModifier recipe = QuantizationModifier(targets="Linear", scheme="NVFP4A16", ignore=["lm_head"])oneshot(model="speakleash/Bielik-11B-v3.0-Instruct", recipe=recipe, output_dir="bielik-nvfp4")speakleash/Bielik-11B-v3.0-Instruct (Apache-2.0).speakleash/Bielik-11B-v3.0-DFlash (Apache-2.0).vllm serve norecyc/Bielik-11B-v3.0-Instruct-NVFP4 \ --served-model-name bielik-11b \ --speculative-config '{"method":"dflash","model":"speakleash/Bielik-11B-v3.0-DFlash","num_speculative_tokens":8}' \ --max-model-len 32768 --gpu-memory-utilization 0.85 \ --host 0.0.0.0 --port 8000from llmcompressor import oneshotfrom llmcompressor.modifiers.quantization import QuantizationModifier recipe = QuantizationModifier(targets="Linear", scheme="NVFP4A16", ignore=["lm_head"])oneshot(model="speakleash/Bielik-11B-v3.0-Instruct", recipe=recipe, output_dir="bielik-nvfp4")