The Friendli Engine supports CUDA-enabled NVIDIA GPUs, which means it relies on a specific version of CUDA and necessitates proper CUDA compute compatibilities. To utilize the Friendli Container effectively, ensure that you have the appropriate NVIDIA GPUs and an NVIDIA driver in place. Currently, we publicly offer a single Friendli Container image (Documentation Index
Fetch the complete documentation index at: https://friendli.ai/docs/llms.txt
Use this file to discover all available pages before exploring further.
registry.friendli.ai/trial:latest) equipped with CUDA 12.4, targeting CUDA compute compatibility versions 8.0, 8.6, 8.9, and 9.0.
To make the right choices regarding GPUs and driver versions, consult the required driver versions and GPUs for the CUDA toolkit and compute compatibility.