What is Friendli Container?
Unmatched Control: Friendli Container provides the Friendli Engine, our cutting-edge serving technology, as a Docker container. This means you can:- Run your own data center or cluster: Deploy the container on your existing GPU machines, giving you complete control over your infrastructure and data security.
- Choose your own cloud provider: If you prefer the cloud, you can still leverage your preferred cloud provider and GPUs.
- Customize your environment: Fine-tune the container configuration to perfectly match your specific needs and workflows.
- Controlled environment: Keep your data within your own environment, ideal for sensitive applications or meeting compliance requirements.
- Unmatched flexibility: Tailor your infrastructure and workflows to your specific needs, pushing the boundaries of AI innovation.
- Cost saving opportunities: Manage your resources on your GPU machines, potentially leading to cost savings compared to cloud-based solutions.
- Data-sensitive users: Securely run your models within your own infrastructure.
- Control enthusiasts: Take full control over your AI environment and workflows.
- Existing cluster owners: Utilize your existing GPU resources for cost-effective generative AI serving.
Getting Started with Friendli Container:
- Generate Your User Token: Visit the Friendli Container page through the Friendli Suite website and generate your unique token.
- Login with Docker Client: Use your token to authenticate with the Docker client on your machine.
- Pull the Friendli Container Image: Run the docker pull command with the provided image name.
- Launch the Friendli Container: Run the docker run command with the desired configuration and credentials.
- Expose Your Model: The container will expose the model for inference.
- Send Inference Requests: Use tools like curl or Python’s requests library to send input prompts or data to the container.