GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ilessio-aiflowlab
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ilessio-aiflowlab
Model Tree
Qwen/Qwen2.5-Coder-14B-Instruct
Input Modalities
Output Modalities
Supported Functionality
anima-vllm:thor-latest engine (vLLM 0.23 · PyTorch
2.11 · CUDA 13), or any vLLM build with NVFP4 (compressed-tensors) support.bash
vllm serve ilessio-aiflowlab/Qwen2.5-Coder-14B-Instruct-NVFP4-anima \ --trust-remote-code --attention-backend TRITON_ATTN \ --gpu-memory-utilization 0.70 --kv-cache-dtype fp8 --max-model-len 32768python
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="x")r = client.chat.completions.create( model="ilessio-aiflowlab/Qwen2.5-Coder-14B-Instruct-NVFP4-anima", messages=[{"role": "user", "content": "Write a Python function to check if a string is a palindrome."}],)print(r.choices[0].message.content)On a Jetson use --runtime nvidia (not --gpus all). The first request JIT-compiles attention (~30–60 s), then it's cached.
| Method | NVFP4 post-training quantization (NVIDIA TensorRT Model Optimizer, NVFP4_DEFAULT_CFG) |
| Format | compressed-tensors (NVFP4 weights + per-group scales; fp8 KV-cache quant) |
| Calibration | 64 diverse prompts (reasoning, code, translation, general) |
| Produced on | NVIDIA Jetson AGX Thor · sm_110a · CUDA 13 · PyTorch 2.11 · via anima-thor-ui |
| Base model | Qwen/Qwen2.5-Coder-14B-Instruct |
General-purpose inference on NVFP4-capable hardware (Blackwell and newer). Quality closely tracks the base model; as with any 4-bit PTQ, expect small deviations on the most numerically sensitive tasks. Inherits the base model's license, capabilities, biases, and intended-use terms — review the base model card. This is an independent community quantization, not affiliated with or endorsed by the base-model authors or NVIDIA.
Released under apache-2.0 (subject to the base model's license). NVIDIA, CUDA, Jetson, and Blackwell are trademarks of NVIDIA Corporation. Quantized by RobotFlow Labs / AIFLOW LABS for the ANIMA edge-AI stack using anima-thor-ui. ⭐ the repo if this is useful.
vllm serve ilessio-aiflowlab/Qwen2.5-Coder-14B-Instruct-NVFP4-anima \ --trust-remote-code --attention-backend TRITON_ATTN \ --gpu-memory-utilization 0.70 --kv-cache-dtype fp8 --max-model-len 32768from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="x")r = client.chat.completions.create( model="ilessio-aiflowlab/Qwen2.5-Coder-14B-Instruct-NVFP4-anima", messages=[{"role": "user", "content": "Write a Python function to check if a string is a palindrome."}],)print(r.choices[0].message.content)