GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
amd
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
amd
Model Tree
moonshotai/Kimi-K2-Thinking
Input Modalities
Output Modalities
Supported Functionality
The model was evaluated on GSM8K benchmarks using the lm-evaluation-harness framework.
The results of GSM8K were obtained using lm-evaluation-harness and latest vLLM.
Launch vLLM
markdown
MODEL_DIR=/data/amd/Kimi-K2-Thinking-W4A8VLLM_ATTENTION_BACKEND="TRITON_MLA" VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0 VLLM_ROCM_USE_AITER_FP4BMM=0 vllm serve $MODEL_DIR \--port 8001 \--trust-remote-code \--gpu-memory-utilization 0.9 \--tensor-parallel-size 8
GSM8K evaluation
markdown
MODEL_ARGS="model=/data/amd/Kimi-K2-Thinking-W4A8,base_url=http://localhost:8001/v1/completions,num_concurrent=999999,timeout=999999,tokenized_requests=False,max_length=38768,temperature=0.6,top_p=0.95,add_bos_token=True,seed=$SEED,trust_remote_code=True"lm_eval \--model local-completions \--model_args $MODEL_ARGS \--tasks gsm8k \--num_fewshot 8 \--batch_size auto
Modifications Copyright(c) 2025 Advanced Micro Devices, Inc. All rights reserved.