GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
kaushikvira
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
kaushikvira
Model Tree
d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
Input Modalities
Output Modalities
Supported Functionality
model.safetensors — 17.1 GiB NVFP4 (W4A4 gs16) text stack + vision tower, W8G32 token embedding + output head; calibrated on 512 Ultrachat samples (seq 2048)model_mtp.safetensors — MTP head (used by the NInfer container; ignored by vLLM)recipe.yaml — the llm-compressor recipe, for reproducibilitybash
pip install -U vllm vllm serve kaushikvira/Qwen3.8-27B-swift-abliterated-NVFP4-HF \ --max-model-len 131072 # 32 GB GPU: 131k fits comfortably; 262k on 48 GB+python
from vllm import LLM, SamplingParamsfrom vllm.sampling_params import GuidedDecodingParams llm = LLM(model="kaushikvira/Qwen3.8-27B-swift-abliterated-NVFP4-HF", max_model_len=131072) schema = { "type": "object", "properties": {"answer": {"type": "string"}, "confidence": {"type": "number"}}, "required": ["answer"],} sp = SamplingParams(temperature=0.0, guided_decoding=GuidedDecodingParams(json=schema))out = llm.chat([{"role": "user", "content": "What is 2+2? Answer in JSON."}], sp)print(out[0].outputs[0].text)The quantization is identical to our NInfer v3 container of the same weights (which carries full gate/needle/llama-benchy numbers on a single RTX 5090), but the vLLM/SGLang-specific numbers for this repo are not measured yet — this is a fresh packaging.
If you run it, please share your numbers in the Community tab (tool + version, GPU, context length, pp/tg tok/s) — we'll collect them here for everyone.
from vllm import LLM, SamplingParamsfrom vllm.sampling_params import GuidedDecodingParams llm = LLM(model="kaushikvira/Qwen3.8-27B-swift-abliterated-NVFP4-HF", max_model_len=131072) schema = { "type": "object", "properties": {"answer": {"type": "string"}, "confidence": {"type": "number"}}, "required": ["answer"],} sp = SamplingParams(temperature=0.0, guided_decoding=GuidedDecodingParams(json=schema))out = llm.chat([{"role": "user", "content": "What is 2+2? Answer in JSON."}], sp)print(out[0].outputs[0].text)