GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
pipecat-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
pipecat-ai
Model Tree
nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16
Input Modalities
Output Modalities
Supported Functionality
Measured recipe (vLLM 0.30, two B200s with tensor parallelism: ~210 tok/s single-stream decode, ~27 ms TTFT, ~5.8k tok/s aggregate at 128 concurrent requests). The ~240 GB of bf16 weights need two 180 GB+ GPUs; B300:2 is the same speed.
bash
vllm serve pipecat-ai/phonellm-max-alpha-2 \ --served-model-name phonellm-max-alpha-2 \ --tensor-parallel-size 2 \ --dtype bfloat16 \ --max-model-len 32768 \ --gpu-memory-utilization 0.90 \ --max-num-seqs 256 \ --kv-cache-dtype fp8 \ --enable-prefix-caching \ --async-scheduling \ --mamba-ssm-cache-dtype float32 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser nemotron_v3--enforce-eager: CUDA graphs are ~7x faster decode on this model.--max-num-seqs 64 --max-cudagraph-capture-size 64 (bf16 KV
cache, also measured) rather than eager mode. fp8 KV only saves memory here (the model has few attention
layers); it does not change outputs measurably.--middleware) rather than editing the template: the reasoning parser must agree
with the template on the mode.PhoneLLM Max Alpha 2 is released under the OpenMDW License Agreement, version 1.1, the same licence as its base model, NVIDIA Nemotron 3.5 Super VL 120B-A12B. If you redistribute this model or your own derivatives of it, you must retain (1) a copy of the OpenMDW License Agreement and (2) the copyright notices and other notices of origin that apply to your distribution. "Nemotron" and "NVIDIA" are trademarks of NVIDIA Corporation, used here only to describe the origin of the base model.