GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
wasifb
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
wasifb
Model Tree
Nanbeige/Nanbeige4.3-33B
Input Modalities
Output Modalities
Supported Functionality
| Scheme | W4A16: int4, symmetric, group size 128, auto_round:auto_gptq packing |
| Quantized | every linear layer in the 52 decoder layers (364) |
| Kept in BF16 | lm_head, embeddings, all norms |
| Tool | auto-round 0.14.2 |
| Tuning | 200 iterations, 512 samples, seqlen 2048, enable_alg_ext=True |
| Calibration | auto-round's default (NeelNanda/pile-10k) |
| Source | Nanbeige/Nanbeige4.3-33B @ c1093c50 |
The recipe is also stored in club3090_build_recipe.json. The tokenizer files, chat template and
generation_config.json are the original model's, byte for byte.
The original checkpoint has no MTP head, so neither does this one.
⚠️ Needs transformers 5.x. config.json keeps the RoPE theta (1.2e8) only in rope_parameters, as
transformers 5 writes it. A stack on transformers 4.x finds no rope_theta and silently falls back to
10,000, which garbles the output. vLLM v0.32.0 (transformers 5.19) and SGLang v0.5.22 (transformers
5.19) both read it correctly.
vLLM (the AutoRound config resolves to vLLM's inc method on its own):
bash
vllm serve wasifb/Nanbeige4.3-33B-AutoRound-W4A16 \ --tensor-parallel-size 2 --max-model-len 204800 --kv-cache-dtype fp8_e4m3 \ --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coderSGLang:
bash
python -m sglang.launch_server --model-path wasifb/Nanbeige4.3-33B-AutoRound-W4A16 \ --tp-size 2 --quantization auto-round --context-length 204800 --kv-cache-dtype fp8_e4m3 \ --chunked-prefill-size 2048 --reasoning-parser qwen3 --tool-call-parser qwen3_coderOn 24 GB cards keep SGLang's --chunked-prefill-size at 2048: 8192 ran out of memory while capturing
the prefill CUDA graphs.
The parsers are the ones the original card uses. Its tool format defaults to XML
(tool_call_format="xml", parsed by qwen3_coder); json works with hermes. Thinking is on by
default (enable_thinking), and preserve_thinking is left to the template, which keeps earlier
reasoning: the original card recommends that for agents and false for chat. Sampling follows the
original card: agentic and tool use temperature=1.0, top_p=0.95, reasoning and chat 0.6 / 0.95.
Tested on 2× RTX 3090 (PCIe), tensor parallel 2, fp8 KV, 204,800 context:
| vLLM v0.32.0 | SGLang v0.5.22 | |
|---|---|---|
| Loads (Marlin int4 kernels) | yes, 9.83 GiB a card | yes, 10.11 GB a card |
| fp8 KV pool | 207,616 tokens | 206,433 tokens |
| verify-full (answers, tool calls, streaming tool calls, reasoning split, a long answer without repetition) | pass | pass |
These are serving checks. Throughput figures and the quality evaluation (club-3090's 8-pack) are pending, and this card will be updated with them.
Nanbeige4.3-33B is licensed under Apache-2.0 (see LICENSE). This quantization is a derivative and
is distributed under the same license.
All credit for the model goes to the Nanbeige team. This repository only adds the quantization.
vllm serve wasifb/Nanbeige4.3-33B-AutoRound-W4A16 \ --tensor-parallel-size 2 --max-model-len 204800 --kv-cache-dtype fp8_e4m3 \ --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coderpython -m sglang.launch_server --model-path wasifb/Nanbeige4.3-33B-AutoRound-W4A16 \ --tp-size 2 --quantization auto-round --context-length 204800 --kv-cache-dtype fp8_e4m3 \ --chunked-prefill-size 2048 --reasoning-parser qwen3 --tool-call-parser qwen3_coder