SK Telecom Elevates LLM Operations with Friendli Dedicated Endpoints
PROBLEM
Running and operating the custom LLMs requires long hours and increases operational costs.
SOLUTION
Leverages Friendli Dedicated Endpoints to serve and operate their LLMs.
RESULT
Onboarding within a few hours, 3x cost savings, and 5x increase in throughput.
NaCloud: Reducing LLM serving costs for a novel writing service.
PROBLEM
Operating a writing service powered by LLMs
Generative AI powered writing service required serving LLMs.
SOLUTION
Use Friendli Container for LLM serving
Friendli Container enabled our client to use Friendli Engine.
RESULT
Cut LLM serving cost instantly
NaCloud was able to cut GPU serving costs.
Upstage: Upstage’s Solar LLMs are operated cost-efficiently without any operation burden, thanks to Friendli Dedicated Endpoints.
PROBLEM
Operated LLMs cost-efficiently under varying input traffic
Upstage needed to manage large language model serving efficiently under varying input traffic.
SOLUTION
Use Friendli Dedicated Endpoints for running LLMs
To solve their problem, Upstage decided to utilize Friendli Dedicated Endpoints which is easy to use for operating large language models.
RESULT
Cost-efficient LLM offering without any operational burden
As a result, Upstage was able to serve their propriety large language model without any operation hassle.
ScatterLab: Zeta blooms with Friendli Engine
PROBLEM
Quality and size of generative model comes with its own cost
The client company wanted their model to produce real-time responses based on current context, which required 17 times more parameters than the original version.
SOLUTION
Use Friendli Engine for Zeta
Scatter Lab adopted Friendli Engine to serve their model. Friendli was able to handle the real-time executions while reducing the cost and the latency dramatically.
RESULT
Reliable service with much improved efficiency
With Friendli Engine, Zeta had launched successfully and is being used in practice. Its enhanced performance of interactive and creative communication is accepting praises while maintaining the cost and latency of the service.
Integration of Friendli Engine with Amazon Sagemaker Jumpstart
PROBLEM
Serving JumpStart Foundation Models incurs performance and cost challenges
It is challenging to serve JumpStart Foundation Models efficiently in Amazon Sagemaker. The models are computationally heavy, incurring high costs and performance problems.
SOLUTION
Friendli Engine has been integrated with Amazon Sagemaker Jumpstart to serve JumpStart Foundation Models
Friendli Engine can be used with NCSOFT VARCO LLMs. Users of VARCO LLMs enjoy high speed and low cost of serving LLMs.
RESULT
Harness the power of Friendli Engine to serve JumpStart Foundation Models
Users can effortlessly utilize NCSOFT VARCO LLMs on Friendli Engine, resulting in cost reduction within Amazon Sagemaker Jumpstart.
Training a Large Language Model (LLM) with Friendli training
PROBLEM
Too much cost for large-scale AI training
Normally, training a large-scale model takes a lot of resources. If you take distributed learning, the burden of faults and loads would only increase.
SOLUTION
Automated and optimized training experience
On Friendli Engine, we could enjoy its special support for distributed learning along with various optimization techniques. Friendli Engine also handled the errors and performance problems to ensure sound training.
RESULT
Made large-scale AI simple
Manipulating Friendli’s automatic matching for state-of-the-art training techniques, training a 13 billion parameter model was felt like a breeze.