GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
auttasak88
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
auttasak88
Model Tree
auttasak88/AgentWorld-35B-A3B-Heretic
Input Modalities
Output Modalities
Supported Functionality
The checkpoint is language-model-only (no vision weights). vLLM detects the compressed-tensors FP8 config automatically.
bash
vllm serve auttasak88/AgentWorld-35B-A3B-Heretic-FP8 \--tensor-parallel-size 2 --language-model-only \--max-model-len 8192 --max-num-seqs 64
--max-num-seqs ≤ ~120 — the linear-attention (GatedDeltaNet) layers use a Mamba-style cache, and a too-high value fails CUDA-graph capture (max_num_seqs exceeds Mamba cache blocks).nvrtc lib dir to LD_LIBRARY_PATH, or force the Marlin FP8 kernel with VLLM_TEST_FORCE_FP8_MARLIN=1.Apache-2.0, inheriting the base model. Abliteration by Heretic; FP8 by llm-compressor; base model by the Qwen team (Qwen/Qwen-AgentWorld-35B-A3B).