GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
peaster1
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
peaster1
Model Tree
zed-industries/zeta-2.1
Input Modalities
Output Modalities
Supported Functionality
vLLM's load-time --quantization fp8 uses one scale per tensor. On this model, that
is enough rounding error to cause degeneration of the model.
| Quant | KLD | Top-1 | MMLU (0-shot) | HellaSwag (acc / acc_norm) | PIQA (acc / acc_norm) | Disk Size | Zed reference prompt |
|---|---|---|---|---|---|---|---|
| BF16 | 0 | 100 % | 37.57 % | 41.16 / 51.80 % | 68.34 / 69.42 % | 16.5 GB | 10/10 |
| FP8-W8A16 (per-channel, this) | 0.00125 | 98.91 % | 37.90 % | 41.03 / 51.58 % | 68.39 / 69.42 % | 9.5 GB | 10/10 |
vLLM --quantization fp8 (per-tensor) | 0.00132 | 98.96 % | — | — | — | 9.5 GB | 0/10 |
This is not a chat model. There is no chat template, only raw /v1/completions with Zed's suffix-prefix-middle prompt format. Zed selects that format by matching the served model name zeta2.1 exactly and sends no temperature, so the server default must be greedy:
bash
vllm serve peaster1/zeta-2.1-FP8-W8A16 \ --served-model-name zeta2.1 \ --override-generation-config '{"temperature":0.0}' \ --speculative-config '{"method":"ngram","num_speculative_tokens":32,"prompt_lookup_max":5,"prompt_lookup_min":3}' \ --enable-prefix-caching --max-model-len 32768