GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ProCreations
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ProCreations
Model Tree
Nanbeige/Nanbeige4.3-33B
Input Modalities
Output Modalities
Supported Functionality
sh
python -m sglang.launch_server \ --model-path ProCreations/Nanbeige4.3-33B-NVFP4-A16 \ --dtype bfloat16 \ --fp4-gemm-backend marlin \ --reasoning-parser nanbeige \ --tool-call-parser nanbeigeThe stock automatic FP4 backend selects Marlin for W4A16_NVFP4 on this GPU; the command makes that choice explicit. Leave --kv-cache-dtype at auto for BF16 KV, or add --kv-cache-dtype fp8_e4m3 for a smaller unit-scale cache. Both settings have stock SM120 load/generation evidence. Independent default-auto smoke passed 4/4: thinking on/off, three weather tool calls and 8,259-token retrieval. Explicit-FP8 raw smoke passed arithmetic, coding, JSON, Chinese and thinking checks. Raw generation smoke is separate from task-accuracy evaluation.
Use the original tokenizer and chat template, including enable_thinking and preserve_thinking. Upstream sampling guidance is temperature 1.0 / top_p 0.95 for agentic/tool work and 0.6 / 0.95 for reasoning/chat.
In SGLang 0.5.21, the model-specific nanbeige parser names alias the Qwen3 reasoning and Qwen3Coder tool detectors. The shared OpenAI-endpoint contract was audited on this project’s FP8 checkpoint with streaming/non-streaming and both aliases; it was not repeated as an A16-specific parser audit. A literal </parameter> inside a tool string can silently truncate structured arguments while raw text stays complete. Validate arguments containing arbitrary code. The shared audit records 44/52 checks passed, with all eight failures attributable to this delimiter limitation. No SGLang patch is applied. See SERVING_PARSERS.md.
Frozen eval-v1 contains 634,268 assistant positions. Thinking, answers, tool calls and assistant history contribute; prompt/system/tool-output tokens remain in context and are excluded from the headline. Measurement uses SGLang native EXTEND/prefill, default-auto/BF16 KV, natural-log KL(BF16 ∥ quant), BF16 top-256 plus one complement bucket. This coarse KL is a lower bound on full-vocabulary KL. Decision labels include think-close, EOS, tool-close, answer-start and annotated reverification starts.
| Domain | Assistant positions | Assistant KL | Decision KL |
|---|---|---|---|
| all | 634,268 | 0.032882 | 0.017419 |
| agentic | 70,490 | 0.007733 | 0.005901 |
| chat | 112,182 | 0.056055 | 0.040037 |
| chinese | 77,925 | 0.049633 | 0.063816 |
| code | 137,335 | 0.019864 | 0.006749 |
| longctx | 5,913 | 0.014842 | 0.004568 |
| math | 115,237 | 0.024032 | 0.016111 |
| multilingual | 47,621 | 0.045523 | 0.017923 |
| stem | 29,823 | 0.059337 | 0.005614 |
| structured | 37,742 | 0.016762 | 0.003745 |
All 298 required core trials are complete. No coding/tools/math regression has a paired prompt-bootstrap CI excluding zero under this release's rule (a coding/tool/math regression whose paired CI excludes zero would block release). agentic_coding_on is borderline: −7.8 pp with CI upper bound 0.0. This does not establish equivalence.
| Suite | A16 passes | BF16 passes | Paired difference (pp) | 95% prompt-bootstrap CI (pp) |
|---|---|---|---|---|
| agentic_coding_on | 43/46 | 46/46 | -7.8 | [-17.8, 0.0] |
| agentic_coding_off | 22/30 | 22/30 | 0.0 | [-20.0, +20.0] |
| tool_off | 51/60 | 47/60 | +6.7 | [-1.7, +15.0] |
| structured | 28/30 | 27/30 | +3.3 | [-10.0, +20.0] |
| math | 55/56 | 54/56 | +0.8 | [-6.7, +8.3] |
| chinese | 44/46 | 43/46 | -3.3 | [-14.4, +6.7] |
| chat | 24/24 | 24/24 | 0.0 | [0.0, 0.0] |
| long_context | 6/6 | 6/6 | 0.0 | [0.0, 0.0] |
Small fixed-seed suites. Paired estimates average seeds within prompts and weight prompts equally; they need not equal differences between pooled trial counts. Confidence intervals cluster prompts. The independent expanded 500-pair length control is complete and reported below.
Exact full-vocabulary think-close probabilities are measured at fresh BF16 thinking positions, excluding assistant history. These are teacher-forced native EXTEND/prefill indicators, not on-policy generated-length predictions. Intervals are 95% conversation-cluster bootstrap CIs.
| Slice | Thinking stop-mass ratio vs BF16 | 95% CI | Positions / conversations |
|---|---|---|---|
| all | 0.995924 | [0.990534, 1.001204] | 395,963 / 252 |
| agentic | 0.986695 | [0.974820, 0.999513] | 33,885 / 53 |
| chinese | 1.011312 | [0.984431, 1.051857] | 32,893 / 20 |
Agentic think-close mass is about 1.3% lower on these fixed histories; its CI lies below 1.0. Both overall and agentic point ratios pass the revised [0.97, 1.03] envelope. The earlier stricter near-unity CI test is an advisory, not the current eligibility rule. Stop shifts remain disclosed rather than described as lossless stopping.
Complete 500/500 fixed pairs across 470 prompts, zero infrastructure errors, native SGLang 0.5.21 / SM120, default-auto/BF16 KV. Request-concurrency ceilings were 32 single-turn and 4 agentic, matching canonical BF16. Prompts, seeds, caps and agent protocol match the reference.
Agentic output-cap hits: 7/60 versus BF16 1/60 (+10 pp, 95% prompt-cluster CI [+3.3, +20]). Agentic restricted-mean length identification bounds are [1.118, 1.172], with 95% envelope [0.909, 1.486]. Overall bounds are [1.019, 1.024], 95% envelope [0.983, 1.063]; overall cap hits are 86 versus 84.
| Domain | Pairs | Token restriction | Restricted-mean ratio or identification bounds | 95% CI / bound envelope | Early censors A16 / BF16 | Cap hits A16 / BF16 | Cap-rate difference95% CI (pp) |
|---|---|---|---|---|---|---|---|
| overall | 500 | 8,192 | [1.019, 1.024] | [0.983, 1.063] | 1 / 4 | 86 / 84 | [-1.45, +2.38] |
| agentic_code | 60 | 8,192 | [1.118, 1.172] | [0.909, 1.486] | 1 / 4 | 7 / 1 | [+3.33, +20.00] |
| chat | 110 | 8,192 | 0.984 | [0.921, 1.048] | 0 / 0 | 18 / 18 | [-4.55, +4.55] |
| chinese | 90 | 8,192 | 0.993 | [0.916, 1.071] | 0 / 0 | 4 / 5 | [-3.33, +0.00] |
| code | 100 | 16,384 | 1.013 | [0.920, 1.123] | 0 / 0 | 10 / 11 | [-6.00, +3.00] |
| math | 140 | 16,384 | 1.014 | [0.990, 1.042] | 0 / 0 | 47 / 49 | [-4.29, +1.43] |
Restricted means concern min(completion length, restriction), not unrestricted completion lengths. Early time/turn stops create identification bounds; no independent-censoring assumption is made. The overall uncertainty interval includes 1.0. The agentic cap increase is significant in this fixed cohort; it is not a proven kernel mechanism or a general-task estimate.
All capped and early-censored rows are retained. Of the seven capped agentic trials, four passed hidden tests, three failed and one had a loop flag. Non-agentic completion checks in this length control are not answer-accuracy grades; the separate core table above reports task accuracy. No unrestricted completion-length or ±3% equivalence is claimed. See LENGTH_500.md and LENGTH_500.json.
Twenty real prompts, five per agentic/code/math/Chinese domain, generated greedily up to 256 tokens with EOS respected. BF16 rescored each exact generated sequence; the first generated token is excluded because it comes from prefill. No hooks, speculation or snapshots were used. The diagnostic uses common native decode/prefill top-20 support plus a complement bucket, with exact BF16 mass on that partition. Different artifacts produce different continuations, so these means do not rank candidates across artifacts and do not measure thinking-stop fidelity.
| Domain | KL(BF16 ∥ decode) | KL(BF16 ∥ prefill) | KL(decode ∥ prefill) |
|---|---|---|---|
| all | 0.021174 | 0.021255 | 0.000792 |
| agentic | 0.004637 | 0.004455 | 0.000415 |
| math | 0.008000 | 0.007957 | 0.000625 |
| code | 0.010708 | 0.010813 | 0.000777 |
| chinese | 0.055956 | 0.056315 | 0.001227 |
This control covers 4,684 decode positions. It is distinct from eval-v1’s fixed-token assistant estimator. No bit-exact equality across batching, graph widths or kernel geometries is claimed.
Stock SGLang 0.5.21 / SM120, FlashInfer, original BF16 head and embeddings, 2,048-token prefill chunks, CUDA decode graphs through batch 128, radix disabled and memory fraction 0.88. Random-token hardware prompts measure speed rather than reasoning quality.
| Checkpoint / cache | 1K decode tok/s | 32K | 128K | 250K | 128-stream aggregate tok/s |
|---|---|---|---|---|---|
| Clean A16, auto / BF16 KV | 70.03 | 54.23 | 32.03 | 21.41 | 1,307.21 |
| Same clean A16, explicit FP8 / unit scales | not measured in this phase | not measured | 41.69 | 30.93 | not measured |
All 19 exact hardware-bank cases passed for auto/BF16 KV. Mean/median timing summaries report 8K TTFT 1.697 s, with input-tokens/TTFT about 4,830 tok/s including first-token work. The long-context FP8 phase covered exact 131,072/256 and 250,000/128 input/output cases; it improves decode throughput but does not establish long-context FP8-cache quality. Relative to the same clean checkpoint with BF16 KV, unit-scale FP8 KV improves decode by about 30% at 128K and 44% at 250K. TTFT/prefill is not improved by those decode gains. Unit-scale and separately calibrated-KV companion speed were effectively equal in that phase. This repo is the clean unit-scale-capable checkpoint.
Auto/BF16-KV speed receipt: ba1f8e14ea81362c7e4b580b81454bdd7498d872a7f5a92a552863ceceb65d74. Long-context FP8 receipt: 688caf4b679cc709deb8eeba3026ca9f86765ad7660de28aa1079e672b0be994. See VEGA_SPEED.md and VEGA_FP8_LONG.md. The native 20-prompt referee uses different prompts and settings and is not a speed comparison.
A separate real-prompt benchmark on Modal RTX PRO 6000 Server Edition (600 W) measured the accepted DFlash2 v1 broad-retrain checkpoint with this exact A16 target. These are measured Modal rates; they are not normalized to the Vega workstation’s 580 W results above.
| Sampling | Matched plain decode tok/s | DFlash2 decode tok/s | Speedup | Emitted tokens per verify |
|---|---|---|---|---|
| Greedy T=0 | 65.42 | 98.87 | 1.51× | 1.898 |
| T=1, top_p=0.95 | 65.15 | 90.71 | 1.39× | 1.769 |
Protocol: 25 real prompts, five each agentic/code/chat/math/Chinese; full-bank warmup, EOS respected and a 256-token output cap. Stock SGLang 0.5.21, explicit --dtype bfloat16, BF16 target/draft KV, batch1 decode graphs, 2,048-token prefix chunks, fixed block8 (anchor plus seven proposals), one parallel draft pass and native stochastic conditional-q rejection. The drafter was trained with an FP8 teacher and was not A16-adapted for this measurement.
This is performance evidence, separate from task-quality parity. Only 7/25 greedy prompts were token-identical between plain and speculative runs; stochastic rejection correctness does not imply bit-exact floating-point paths. No accuracy or generated-length equivalence is inferred from this speed test.
The companion draft model is Nanbeige4.3-33B-DFlash2. See DFLASH_A16_SPEED.json for the measured rows and immutable identities.
Base revision: c1093c501cc6460427c0ac634db7223d0450c0ee. This A16 checkpoint was independently calibrated with the weight-only NVFP4A16 scheme. FP4 E2M1 weights use groups of 16 with E4M3 block scales and a global weight scale; activations, embeddings and output head remain BF16. It does not reuse W4A4 activation quantization.
GPTQ uses block 128, dampening 0.01 and static activation ordering. Importance-weighted MSE scale search uses imatrix_mse, grid 40, squared-error norm 2, max-shrink 0.5, patience 5 and strict importance statistics. Export converts compressed-tensors names/global-scale reciprocals to SGLang’s modelopt / W4A16_NVFP4 loader convention, preserving packed FP4 values and block scales. This is a serialization conversion, not a claim that NVIDIA ModelOpt performed quantization.
Quantizer environment: llm-compressor 0.13.0, compressed-tensors 0.18.0, Transformers 5.12.1, PyTorch 2.13.0+cu130. Serving uses the stock image lmsysorg/sglang:v0.5.21, SGLang commit e00930c5489053f26d86b179cee0d087f846acbb, FlashInfer 0.6.18 and sglang-kernel 0.4.7.
Frozen calib-v1 contains 1,213 EOS-complete BF16-weight/BF16-KV conversations and 4,054,730 tokens. Source token mix is approximately 35% agentic coding, 15% other code, 12% math, 12% chat, 8% Chinese, 5% multilingual, 5% structured/tool, 5% long context and 3% STEM. Selected calibration uses 556 native windows / 1,086,255 tokens, seed 43, normally 2K with retained long windows up to 32K. No eval data is used for calibration or sensitivity allocation.
Calibration manifest SHA256: 4305272fa895081ac90038f656cf3eec2e40cedf17edc3110e459304abc12191.
Eval-v1 manifest SHA256: 4fe31e47a0b7426e1e3f89fa4945343561681f5112ef125cd43958cac3556fe5.
Quantization took 3,140.29 s on Modal RTX PRO 6000; the A16 format export took 29.64 s. The clean copy removes all 104 KV-scale tensors and the KV scheme, taking 135.36 s on CPU while preserving weights and linear scales. Tensor bytes: 22,070,230,448. See QUANT_CARD.md.
Apache-2.0, inherited from the base model. See LICENSE. Weight precision and serialization change; the original BF16 head/embeddings remain.
The project's FP8 checkpoint and DFlash2 draft model are published; EXL3 is also published.
python -m sglang.launch_server \ --model-path ProCreations/Nanbeige4.3-33B-NVFP4-A16 \ --dtype bfloat16 \ --fp4-gemm-backend marlin \ --reasoning-parser nanbeige \ --tool-call-parser nanbeige