GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
akalakaaaa
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
akalakaaaa
Model Tree
Qwen/Qwen2.5-Coder-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Install an NVIDIA CUDA-enabled PyTorch build suitable for your GPU, then python -m pip install -r requirements-inference.txt in a separate environment. The requirements record the historical Windows inference stack (PyTorch 2.8.0+cu128); other stacks have not been validated for this release.
After downloading this repository to ./base, run:
bash
python base/inference.py --base ./base --offline --max-new-tokens 512For base plus adapter downloaded to sibling folders, run the same script with --adapter ./adapter. The loader reads the quantization configuration already stored in the base; do not quantize it again. Allocate enough GPU memory for model weights, activations and KV cache, not just the on-disk weight size.
The paired historical RTL experiment (2026-09-09) reported 50/156 full passes for this base and 61/156 with the step-80 adapter, with 18 gains and 7 regressions. This was a public development set on NVIDIA / Vivado 2018.3, and is not an AMD/ROCm competition result. Full result metadata is included in the adapter repository. Model loading or inference has not been re-run as part of the 2026-10-24 packaging checks.
Apache-2.0; preserve the included LICENSE and attribution. Weights and upstream model/tokenizer configurations are unchanged. Packaging adds this model card, inference example, attribution and provenance. A locally missing merges.txt was restored byte-for-byte from the same pinned upstream revision. See provenance.json.