Use Cases
Integration of PeriFlow with Amazon Sagemaker Jumpstart
PeriFlow supports running NCSOFT VARCO LLMs in Amazon Sagemaker Jumpstart.
Amazon Sagemaker Jumpstart users can run VARCO LLMs with PeriFlow, FriendliAI’s cutting-edge generative AI engine. This opens a door to integrate other Jumpstart foundation models with PeriFlow.
Problem
Serving JumpStart Foundation Models incurs performance and cost challenges.
It is challenging to serve JumpStart Foundation Models efficiently in Amazon Sagemaker. The models are computationally heavy, incurring high costs and performance problems.
Solution
PeriFlow has been integrated with Amazon Sagemaker Jumpstart to serve JumpStart Foundation Models.
PeriFlow can be used with NCSOFT VARCO LLMs. Users of VARCO LLMs enjoy high speed and low cost of serving LLMs.
Result
Harness the power of PeriFlow to serve JumpStart Foundation Models
Users can effortlessly utilize NCSOFT VARCO LLMs on PeriFlow, resulting in cost reduction within Amazon Sagemaker Jumpstart.
TUNiB’s emotional chatbots with PeriFlow cloud
Launch diverse Generative AI models, managed by PeriFlow cloud
TUNiB's emotional chatbot services are earning accolades with PeriFlow cloud - FriendliAI's managed service for serving LLM.
Problem
Managing multiple AI models incurs significant time and costs
The client required to oversee the deployment of various Generative AI models to manage unpredictable real-time requests.
Solution
Use PeriFlow cloud for various models
PeriFlow cloud has enabled TUNiB to handle real-time executions with ease while also managing the number of deployments necessary to minimize operation costs. In addition, PeriFlow has further improved its model deployments, significantly reducing both costs and latency.
Result
Convenience and dependable service without the need for self-management
With PeriFlow cloud, TUNiB is able to deliver enhanced performance in interactive and creative AI models, all while maintaining low operating costs and request latency.
Lee Luda (이루다) 2.0 blooms with PeriFlow
Realize efficient Generative AI with PeriFlow
Scatter Lab's renewed chatbot service is accepting praise with PeriFlow, FriendliAI's LLM serving engine that speeds up Generative AI.
Problem
Quality and size of generative model comes with its own cost
The client company wanted their model to produce real-time responses based on current context, which required 17 times more parameters than the original version.
Solution
Use PeriFlow for Lee Luda 2.0
Scatter Lab adopted PeriFlow to serve their model. PeriFlow was able to handle the real-time executions while reducing the cost and the latency dramatically.
Result
Reliable service with much improved efficiency
With PeriFlow, Lee Luda 2.0 had launched successfully and is being used in practice. Its enhanced performance of interactive and creative communication is accepting praises while maintaining the cost and latency of the service.
Training a Large Language Model (LLM) with PeriFlow training
Swift and Sound; develop your own large-scale AI with PeriFlow training
We developed a GPT-3 13B model to show what it's like to train a LLM on PeriFlow training.
Problem
Too much cost for large-scale AI training
Normally, training a large-scale model takes a lot of resources. If you take distributed learning, the burden of faults and loads would only increase.
Solution
Automated and optimized training experience
On PeriFlow, we could enjoy its special support for distributed learning along with various optimization techniques. PeriFlow also handled the errors and performance problems to ensure sound training.
Result
Made large-scale AI simple
Manipulating PeriFlow’s automatic matching for state-of-the-art training techniques, training a 13 billion parameter model was felt like a breeze.
PeriFlow
Use PeriFlow for your generative language model
A client company using Transformer-based models was in a difficult situation, and we could help them with our efficient serving system.
Problem
Resource-consuming generative models
Transformer-based generative models take massive cost for serving, making companies reluctant to use them.
Solution
Tens of times increased efficiency using PeriFlow
We offered PeriFlow, our new serving system for generative models. With PeriFlow, the generative model of the company was able to experience much improves throughput with low and steady latency.
Result
More affordable use of generative models
The company can use the models without cost concerns. PeriFlow can be helpful to any kind of generative models in need of high performance with low cost.
