GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Wondernutts
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Wondernutts
Model Tree
Nimbz/Gemma-4-Gembrain-X-Core-31B
Input Modalities
Output Modalities
Supported Functionality
Nimbz built Gembrain-X-Core to combine varied roleplay and creative writing with character consistency, instruction following, reasoning, and structured image-prompt writing. Upstream describes thinking-off as a quicker generalist mode and thinking-on as more tightly focused on character traits and harder tasks, with longer reasoning and potentially less variation. These describe the source model's intended behavior, not a measured quality guarantee for this quantization.
| Item | Details |
|---|---|
| Format | Compressed-tensors NVFP4; not GGUF, AWQ, or OpenVINO IR |
| Weight shards | Approximately 19.04 GiB / 20.45 GB; not total runtime VRAM |
| Intended hardware | NVIDIA Blackwell with a compatible native NVFP4 backend |
| Runtime | vLLM with Gemma 4 and compressed-tensors NVFP4 support |
| Validation scope | Text-only runtime checks; no full-context or source-BF16 quality-parity claim |
| KV cache | Not quantized by this export; the example below uses BF16 |
Native NVFP4 support is backend- and hardware-dependent. Other hardware combinations are not established here. This is not the Intel/OpenVINO checkpoint, and Intel Arc/B70 benchmark figures do not apply. Runtime memory also includes KV cache, activations, workspaces, and framework overhead; weight-file size is not a VRAM requirement.
The export uses an expanded importance-matrix weight observer with 128 calibration sequences, maximum sequence length 2,048, and 138,503 calibration tokens. Conversion versions: llmcompressor 0.14.0, compressed-tensors 0.19.0, Transformers 5.16.1, and PyTorch 2.13.0+cu130.
Quantized text linear layers use FP4 weights in groups of 16, dynamic FP4 activations, and FP8 scale metadata. The output head and vision components remain at higher precision. Packed FP4 storage can appear as byte-sized tensors; that does not make these INT8 weights. The export does not quantize the KV cache.
Existing text-only checks selected FlashInferCutlassNvFp4LinearKernel and used BF16 KV cache. Retaining source vision modules does not establish working or validated vision/audio inference. Use text-only serving unless you separately validate a multimodal path.
Install a compatible build using the official vLLM installation guide. The following is a conservative starting example, not a newly benchmarked deployment profile. It limits total context to 4,096 tokens and concurrency to one sequence; memory fit still depends on your GPU and software build.
bash
vllm serve Wondernutts/Gemma-4-Gembrain-X-Core-31B-NVFP4 \ --served-model-name Gemma-4-Gembrain-X-Core-31B-NVFP4 \ --quantization compressed-tensors \ --dtype bfloat16 \ --kv-cache-dtype bfloat16 \ --max-model-len 4096 \ --max-num-seqs 1 \ --gpu-memory-utilization 0.85 \ --enforce-eager \ --language-model-only \ --skip-mm-profiling \ --generation-config vllm \ --reasoning-parser gemma4 \ --default-chat-template-kwargs '{"enable_thinking":false}' \ --host 127.0.0.1Use the included tokenizer and chat template. Do not substitute a tokenizer from an unrelated model.
bash
curl http://127.0.0.1:8000/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model":"Gemma-4-Gembrain-X-Core-31B-NVFP4", "messages":[{"role":"user","content":"Write a short scene in which an innkeeper recognizes an old friend but keeps it quiet."}], "temperature":0.95, "top_p":0.95, "top_k":-1, "min_p":0.05, "max_tokens":256, "chat_template_kwargs":{"enable_thinking":false} }'The sampler values follow Nimbz's stated starting setup. Upstream ranges are temperature 0.7–1.0, top-p 0.95–1.0, min-p 0.03–0.05, with top-k disabled (-1 in vLLM). Optional Adaptive-P settings mentioned upstream are not assumed to be available in this server.
To request thinking, change the request's enable_thinking to true while keeping the Gemma 4 template and reasoning parser. Allow a larger output budget for reasoning plus the final answer, and have the client display the final answer separately from reasoning.
Nimbz created the merge. Google DeepMind provided the Gemma base. Source ingredients credit Danielbrdz, Jackrong, p-e-r-e-g-r-i-n-e, ReadyArt, ConicCat, AuriAetherwiing, llmfan46, zerofata, nbeerbower, LatitudeGames, BirdToast, Green-eyedDevil, bgg1996, sophosympatheia, Blazed-Forge, and trohrbaugh. Upstream also credits ArceeAI/mergekit and zerofata's Gemma merge support. Wondernutts performed this NVFP4 conversion.
Converted source revision: e8d9a041b7d70854f665aa4a171c29b480c0b1b3.
See the source card at that revision and the preserved upstream card for the complete merge recipe and acknowledgements. The source declares Apache-2.0; see LICENSE and NOTICE.
vllm serve Wondernutts/Gemma-4-Gembrain-X-Core-31B-NVFP4 \ --served-model-name Gemma-4-Gembrain-X-Core-31B-NVFP4 \ --quantization compressed-tensors \ --dtype bfloat16 \ --kv-cache-dtype bfloat16 \ --max-model-len 4096 \ --max-num-seqs 1 \ --gpu-memory-utilization 0.85 \ --enforce-eager \ --language-model-only \ --skip-mm-profiling \ --generation-config vllm \ --reasoning-parser gemma4 \ --default-chat-template-kwargs '{"enable_thinking":false}' \ --host 127.0.0.1curl http://127.0.0.1:8000/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model":"Gemma-4-Gembrain-X-Core-31B-NVFP4", "messages":[{"role":"user","content":"Write a short scene in which an innkeeper recognizes an old friend but keeps it quiet."}], "temperature":0.95, "top_p":0.95, "top_k":-1, "min_p":0.05, "max_tokens":256, "chat_template_kwargs":{"enable_thinking":false} }'