GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Wondernutts
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Wondernutts
Model Tree
Nimbz/Schattenblume-31B
Input Modalities
Output Modalities
Supported Functionality
Schattenblume is a roleplay-first merge of scotoma-2, Giftige-Blume v1, and MeroMero v2 over Gemma 4 31B IT. Nimbz's stated goals are stronger character agency, varied prose, and less repetitive stock Gemma phrasing. It is intended to follow the character as written rather than automatically agree with the user. These are upstream design goals, not a measured quality guarantee for this quantization.
| Item | Details |
|---|---|
| Format | Compressed-tensors NVFP4; not GGUF, AWQ, or OpenVINO IR |
| Weight shards | Approximately 19.04 GiB / 20.45 GB; not total runtime VRAM |
| Intended hardware | NVIDIA Blackwell with a compatible native NVFP4 backend |
| Runtime | vLLM with Gemma 4 and compressed-tensors NVFP4 support |
| Validation scope | Text-only runtime checks; no full-context or source-BF16 quality-parity claim |
| KV cache | Not quantized by this export; the example below uses BF16 |
Native NVFP4 support is backend- and hardware-dependent. Other hardware combinations are not established here. This is not the Intel/OpenVINO checkpoint, and Intel Arc/B70 benchmark figures do not apply. Runtime memory also includes KV cache, activations, workspaces, and framework overhead; weight-file size is not a VRAM requirement.
The export uses an expanded importance-matrix weight observer with 128 calibration sequences, maximum sequence length 2,048, and 138,503 calibration tokens. Conversion versions: llmcompressor 0.14.0, compressed-tensors 0.19.0, Transformers 5.16.1, and PyTorch 2.13.0+cu130.
Quantized text linear layers use FP4 weights in groups of 16, dynamic FP4 activations, and FP8 scale metadata. The output head and vision components remain at higher precision. Packed FP4 storage can appear as byte-sized tensors; that does not make these INT8 weights. The export does not quantize the KV cache.
Existing text-only checks selected FlashInferCutlassNvFp4LinearKernel and used BF16 KV cache. Retaining source vision modules does not establish working or validated vision/audio inference. Use text-only serving unless you separately validate a multimodal path.
Install a compatible build using the official vLLM installation guide. The following is a conservative starting example, not a newly benchmarked deployment profile. It limits total context to 4,096 tokens and concurrency to one sequence; memory fit still depends on your GPU and software build.
bash
vllm serve Wondernutts/Schattenblume-31B-NVFP4 \ --served-model-name Schattenblume-31B-NVFP4 \ --quantization compressed-tensors \ --dtype bfloat16 \ --kv-cache-dtype bfloat16 \ --max-model-len 4096 \ --max-num-seqs 1 \ --gpu-memory-utilization 0.85 \ --enforce-eager \ --language-model-only \ --skip-mm-profiling \ --generation-config vllm \ --reasoning-parser gemma4 \ --default-chat-template-kwargs '{"enable_thinking":false}' \ --host 127.0.0.1Use the included tokenizer and chat template. Do not substitute a tokenizer from an unrelated model.
bash
curl http://127.0.0.1:8000/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model":"Schattenblume-31B-NVFP4", "messages":[{"role":"user","content":"Write a short scene in which an innkeeper recognizes an old friend but keeps it quiet."}], "temperature":0.9, "top_p":0.95, "top_k":-1, "min_p":0.05, "repetition_penalty":1.0, "max_tokens":256, "chat_template_kwargs":{"enable_thinking":false} }'These values are within Nimbz's suggested ranges: temperature 0.8–1.1, top-p 0.95, min-p 0.05–0.15, top-k disabled (-1 in vLLM), and repetition penalty off (1.0). Optional Adaptive-P settings mentioned upstream are not assumed to be available in this server. A clear character description, place, and mood give the model more to work with.
Upstream recommends thinking for context-heavy scenes. To request it, change enable_thinking to true while keeping the Gemma 4 template and reasoning parser. Allow a larger output budget for reasoning plus the final answer, and have the client display them separately. This advice does not establish an untested long-context capacity.
Nimbz created Schattenblume. Google DeepMind provided Gemma 4. Its direct ingredients are ReadyArt's scotoma-2, Blazed-Forge's Giftige-Blume v1, and zerofata's MeroMero v2. Upstream acknowledges AesSedai/ReadyArt for scotoma, zerofata for Gemma merge support, and ArceeAI/mergekit. Wondernutts performed this NVFP4 conversion.
Converted source revision: 9d1a417e793a4de38ac918685c107d47655d1b03.
See the source card at that revision and the preserved upstream card for the complete merge recipe and acknowledgements. The source declares Apache-2.0; see LICENSE and NOTICE.
vllm serve Wondernutts/Schattenblume-31B-NVFP4 \ --served-model-name Schattenblume-31B-NVFP4 \ --quantization compressed-tensors \ --dtype bfloat16 \ --kv-cache-dtype bfloat16 \ --max-model-len 4096 \ --max-num-seqs 1 \ --gpu-memory-utilization 0.85 \ --enforce-eager \ --language-model-only \ --skip-mm-profiling \ --generation-config vllm \ --reasoning-parser gemma4 \ --default-chat-template-kwargs '{"enable_thinking":false}' \ --host 127.0.0.1curl http://127.0.0.1:8000/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model":"Schattenblume-31B-NVFP4", "messages":[{"role":"user","content":"Write a short scene in which an innkeeper recognizes an old friend but keeps it quiet."}], "temperature":0.9, "top_p":0.95, "top_k":-1, "min_p":0.05, "repetition_penalty":1.0, "max_tokens":256, "chat_template_kwargs":{"enable_thinking":false} }'