What is Friendli Container
Unmatched control: Friendli Container provides the Friendli Engine, our cutting-edge serving technology, as a Docker container. With it, you can:- Run your own data center or cluster: Deploy the container on your existing GPU machines, giving you complete control over your infrastructure and data security.
- Choose your own cloud provider: If you prefer the cloud, you can still leverage your preferred cloud provider and GPUs.
- Customize your environment: Fine-tune the container configuration to perfectly match your specific needs and workflows.
- Controlled environment: Keep your data within your own environment, ideal for sensitive applications or meeting compliance requirements.
- Unmatched flexibility: Tailor your infrastructure and workflows to your specific needs, pushing the boundaries of AI innovation.
- Cost saving opportunities: Manage your resources on your GPU machines, potentially leading to cost savings compared to cloud-based solutions.
- If you handle sensitive data: Securely run your models within your own infrastructure.
- If you want full control: Take charge of your AI environment and workflows.
- If you already own a GPU cluster: Utilize your existing GPU resources for cost-effective generative AI serving.
Getting started with Friendli Container
- Generate Your Personal API Key: Visit the Friendli Container page through the Friendli Suite website and generate your Personal API Key.
- Login with Docker Client: Use your API Key to authenticate with the Docker client on your machine.
- Pull the Friendli Container Image: Run the docker pull command with the provided image name.
- Launch the Friendli Container: Run the docker run command with the desired configuration and credentials.
- Expose Your Model: The container will expose the model for inference.
- Send Inference Requests: Use tools like curl or Python’s requests library to send input prompts or data to the container.