GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Wondernutts
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Wondernutts
Model Tree
Cyclone-Labs/Starless-Seraph-31B
Input Modalities
Output Modalities
Supported Functionality
| Detail | Value |
|---|---|
| Original checkpoint | Cyclone-Labs/Starless-Seraph-31B |
| Source revision | a7b4bdc9ee204a2a7bd05f473013c29db4955014 |
| Weight format | NVFP4, compressed-tensors |
| Weight shard size | 20.45 GB; runtime memory also includes cache and workspace |
| Quantization | Four-bit floating-point weights; dynamically quantized input activations; group size 16 with FP8 scales |
| Calibration | 128 synthetic roleplay samples, up to 2,048 tokens per sample; 138,503 tokens total |
| Weight observer | nvfp4_expanded_imatrix, strict mode |
| Conversion libraries | llmcompressor 0.14.0; compressed-tensors 0.19.0; Transformers 5.16.1; PyTorch 2.13.0+cu130 |
| KV cache | Left unquantized during conversion; BF16 used for serving validation |
Vision/audio modules, embeddings, routers, and lm_head were excluded from weight quantization. The source tokenizer and chat template are included. The NVFP4 conversion changes the stored weight representation; it adds no new fine-tuning.
Validated with vLLM 0.29.0 on an NVIDIA RTX PRO 6000 Blackwell 96 GB. Use a runtime and GPU that support native NVFP4. These are the settings used for the text-only serving checks:
bash
vllm serve Wondernutts/Starless-Seraph-31B-NVFP4 \ --served-model-name Starless-Seraph-31B-NVFP4 \ --quantization compressed-tensors \ --dtype bfloat16 \ --max-model-len 32768 \ --gpu-memory-utilization 0.88 \ --kv-cache-dtype bfloat16 \ --max-num-seqs 4 \ --max-num-batched-tokens 8192 \ --enable-chunked-prefill \ --enforce-eager \ --language-model-only \ --skip-mm-profiling \ --generation-config vllm \ --chat-template-content-format string \ --reasoning-parser gemma4 \ --default-chat-template-kwargs '{"enable_thinking": false}'Use the included chat template. Set enable_thinking to true to request thinking mode. Both modes were included in compatibility checks; sampling settings and the choice of thinking mode remain application choices.
The upstream repository declares Apache-2.0. A copy is included in LICENSE. The original model card is reproduced below with credit to its authors; relative asset links were changed to point to the original repository. Upstream descriptions and recommendations refer to the original model.
vllm serve Wondernutts/Starless-Seraph-31B-NVFP4 \ --served-model-name Starless-Seraph-31B-NVFP4 \ --quantization compressed-tensors \ --dtype bfloat16 \ --max-model-len 32768 \ --gpu-memory-utilization 0.88 \ --kv-cache-dtype bfloat16 \ --max-num-seqs 4 \ --max-num-batched-tokens 8192 \ --enable-chunked-prefill \ --enforce-eager \ --language-model-only \ --skip-mm-profiling \ --generation-config vllm \ --chat-template-content-format string \ --reasoning-parser gemma4 \ --default-chat-template-kwargs '{"enable_thinking": false}'