Three ways to run generative AI models with Friendli Engine:
01
Friendli Container
Serve LLMs/LMMs inferences with Friendli Engine in your GPU environment
Learn more02
Friendli Dedicated Endpoints
Build and run LLMs/LMMs on autopilot with Friendli Dedicated Endpoints
Learn more03
Friendli Serverless Endpoints
Fast and affordable API for open-source generative AI models
Learn more
NextDay AI
LLM-powered chatbot company cuts GPU costs by more than 50% instantly.
Problem
High H100 GPU costs from processing ~0.5 trillion tokens per month.
Solution
Use Friendli Container for LLM serving
Result
Costs were instantly cut by more than 50%.
SK Telecom Elevates LLM Operations with Friendli Dedicated Endpoints
SKT’s custom LLMs were deployed seamlessly with Friendli Dedicated Endpoints, achieving 5x throughput and reducing 3x operational costs.
Problem
Running and operating the custom LLMs requires long hours and increases operational costs.
Solution
Leverages Friendli Dedicated Endpoints to serve and operate their LLMs.
Result
Onboarding within a few hours, 3x cost savings, and 5x increase in throughput.
Reducing LLM serving costs for a novel writing service
Friendli Container helped NaCloud reduce the cost of serving LLMs.
Problem
Operating a LLM writing service requires high inference cost
Solution
Use Friendli Container for LLM serving
Result
Cut LLM serving cost instantly
Upstage LLMs with Friendli Dedicated Endpoints
Upstage’s Solar LLMs are operated cost-efficiently without any operation burden, thanks to Friendli Dedicated Endpoints.
Problem
Translation traffic inference (~100k/day) needs cost efficient operation
Solution
Use Friendli Dedicated Endpoints for running LLMs
Result
Cost-efficient LLM offering without any operational burden
TUNiB’s emotional chatbots with Friendli Dedicated Endpoints
TUNiB’s chatbot operates LLM inference requests smoothly with Friendli Dedicated Endpoints.
Problem
Managing chatbot LLMs incurs significant engineering effort
Solution
Use Friendli Dedicated Endpoints for the models
Result
Convenient, reliable, and cost-efficient service without the need for self-management