GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
IndexTeam
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
IndexTeam
Model Tree
IndexTeam/Index-Translate-35B-A3B-preview
Input Modalities
Output Modalities
Supported Functionality
NVFP4 (W4A4: 4-bit floating-point weights with per-group-16 scales, 4-bit floating-point activations with calibrated per-tensor global scales), produced with NVIDIA Model Optimizer 0.44 (modelopt format, see hf_quant_config.json). Calibrated on a small bilingual translation corpus.Linear layers of the language model are quantized; lm_head, embeddings, MoE router gates and shared-expert gates are kept in BF16. MTP (multi-token prediction) weights are preserved.hf_quant_config.json - serve with vLLM (--quantization modelopt, auto-detected in recent vLLM).Measured on the quantized model in memory on an NVIDIA A100 before export against the original BF16 checkpoint (greedy decoding, official translation prompt):
| Metric | BF16 | FP4 | Delta |
|---|---|---|---|
| Perplexity (fixed corpus) | 2.5621 | 2.6039 | +1.63% |
| zh->en generation identical | - | - | yes |
| en->zh generation identical | - | - | yes |
bash
vllm serve IndexTeam/Index-Translate-35B-A3B-preview-FP4 --host 127.0.0.1 --port 8000 --max-model-len 4096Hardware note: full NVFP4 (W4A4) acceleration requires an NVIDIA Blackwell GPU (SM100+, e.g. B200 / RTX 50 series). On older GPUs (Hopper / Ampere) vLLM loads the checkpoint with weight-only dequantization - memory is still reduced, but there is no FP4 compute speedup. For non-Blackwell serving we recommend the FP8 build.
Translation prompt format (greedy decoding, temperature=0 recommended; chat template with enable_thinking: false):
text
请将以下文本翻译为{target-language}，直接输出翻译结果，不要进行任何解释。 {source-text}Beyond plain translation, the models follow the instTrans constrained-translation format. The official client wraps requests into the canonical structure 【源文】<text> + numbered 1. 【硬性要求】<hard constraints> + 2. 【注意】<soft constraints> + suffix instructions:
碳纤维:carbon fiber, 抗裂缝:crack resistance), and format/structure preservation for JSON/CSV/code/placeholders.Full prompt reference: github.com/bilibili/Index-Translate (Instruction Following section, docs/prompts.md, inference/llm/cases/).
See the base model card for the full instTrans constrained-translation format and serving presets. GGUF builds for local inference are published in Index-Translate-35B-A3B-preview-GGUF, and an FP8 build for Hopper/Ampere serving in Index-Translate-35B-A3B-preview-FP8.
Quantized and published by the Index team, 2026-10-04.
vllm serve IndexTeam/Index-Translate-35B-A3B-preview-FP4 --host 127.0.0.1 --port 8000 --max-model-len 4096请将以下文本翻译为{target-language}，直接输出翻译结果，不要进行任何解释。{source-text}