GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Wondernutts
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Wondernutts
Model Tree
DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
Input Modalities
Output Modalities
Supported Functionality
| Item | Details |
|---|---|
| Pinned source revision | fd6a26869dc5775b3c81ce65468af634dd300b02 |
| Actual architecture | Qwen3_5ForConditionalGeneration, model type qwen3_5 |
| Format | ModelOpt MIXED_PRECISION, NVFP4 plus FP8 |
| Weight files | Approximately 21.33 GiB total, excluding runtime allocations |
| MLP projections | 192 NVFP4 layer entries, group size 16 |
| Selected attention projections | 208 FP8 layer entries |
| Remaining components | Retained at higher precision |
| KV cache | No KV-cache quantization recipe in the export |
| Multimodal status | Components/configuration retained; not validated by this card |
The name follows the upstream Qwen3.8 branding. The stored architecture remains Qwen3_5ForConditionalGeneration; this is not a reason to replace the tokenizer or change model_type. Use the included tokenizer, configuration and chat template together.
Weight-file size is not the VRAM requirement. KV cache, recurrent state, activations and runtime workspace need additional memory. A 24 GB card is not guaranteed to fit this model with a useful context window. This is not OpenVINO IR or GGUF, and Intel Arc/OpenVINO or llama.cpp benchmark numbers do not apply to this export.
The conversion used NVIDIA Model Optimizer with local-Hessian calibration and an FP8 scale sweep. The export records ModelOpt 0.47.0. Its configuration contains per-layer quantized_layers entries: NVFP4 for the MLP gate/up/down projections, FP8 for selected linear-attention and full-attention projections, and higher precision elsewhere. NVFP4 is a floating-point format, not INT4/AWQ.
This mixed layout is intentional. Do not overwrite its metadata with a uniform FP4 configuration or force the compressed-tensors loader. The calibration sample/token counts are not recorded in the retained card metadata, so the Gemma conversions' calibration counts are not claimed for this model.
Use a CUDA-compatible vLLM build supporting this Qwen architecture and ModelOpt mixed precision. Consult the vLLM installation guide, Qwen architecture recipe and ModelOpt mixed-precision loader reference.
vLLM's loader identifies MIXED_PRECISION as modelopt_mixed and uses the per-layer algorithm map. That loader support alone does not establish compatibility or quality for this particular export. The command below is a configuration to validate, not a newly tested benchmark or a supported-runtime guarantee:
bash
vllm serve Wondernutts/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-NVFP4 \ --served-model-name qwen-turbo-nvfp4 \ --quantization modelopt_mixed \ --dtype bfloat16 \ --kv-cache-dtype bfloat16 \ --max-model-len 4096 \ --max-num-seqs 1 \ --gpu-memory-utilization 0.85 \ --enforce-eager \ --language-model-only \ --skip-mm-profiling \ --generation-config vllm \ --reasoning-parser qwen3 \ --default-chat-template-kwargs '{"enable_thinking":false}' \ --host 127.0.0.1Blackwell is the intended native-NVFP4 hardware path. Other GPUs depend on backend/fallback support and are not validated here. If the runtime rejects modelopt_mixed, a layer scheme, or a tensor shape, check that runtime's support rather than changing the model's architecture or quantization tags to force loading. Leave the endpoint local until access control is configured.
The 4,096-token limit includes input plus output and is a starting budget, not the model's maximum. Eager mode prioritizes initial bring-up, not maximum throughput. Raise context/concurrency only after checking memory use and correctness.
Once the server successfully loads, try:
bash
curl http://127.0.0.1:8000/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model": "qwen-turbo-nvfp4", "messages": [{"role":"user","content":"A traveler enters a snowy mountain inn with a sealed letter. Write a concise opening scene without revealing the letter contents."}], "max_tokens": 512, "temperature": 1.0, "top_p": 0.95, "top_k": 20, "repetition_penalty": 1.0, "chat_template_kwargs": {"enable_thinking": false} }'Temperature, top-p and top-k above match the included generation defaults, not an independently optimized RP preset. The included template also supports enable_thinking: true and reasoning_effort values low, medium or xhigh. For a reasoning trial, use {"enable_thinking":true,"reasoning_effort":"low"} and a larger output budget within the total context limit. Without an explicit effort value, the template defaults to xhigh, which may consume substantially more reasoning tokens. Keep reasoning separate from the displayed final answer.
Browse the Wondernutts NVFP4 collection.
vllm serve Wondernutts/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-NVFP4 \ --served-model-name qwen-turbo-nvfp4 \ --quantization modelopt_mixed \ --dtype bfloat16 \ --kv-cache-dtype bfloat16 \ --max-model-len 4096 \ --max-num-seqs 1 \ --gpu-memory-utilization 0.85 \ --enforce-eager \ --language-model-only \ --skip-mm-profiling \ --generation-config vllm \ --reasoning-parser qwen3 \ --default-chat-template-kwargs '{"enable_thinking":false}' \ --host 127.0.0.1curl http://127.0.0.1:8000/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model": "qwen-turbo-nvfp4", "messages": [{"role":"user","content":"A traveler enters a snowy mountain inn with a sealed letter. Write a concise opening scene without revealing the letter contents."}], "max_tokens": 512, "temperature": 1.0, "top_p": 0.95, "top_k": 20, "repetition_penalty": 1.0, "chat_template_kwargs": {"enable_thinking": false} }'