Use Cases

usecases_05

Integration of PeriFlow with Amazon Sagemaker Jumpstart

PeriFlow supports running NCSOFT VARCO LLMs in Amazon Sagemaker Jumpstart.

Amazon Sagemaker Jumpstart users can run VARCO LLMs with PeriFlow, FriendliAI’s cutting-edge generative AI engine. This opens a door to integrate other Jumpstart foundation models with PeriFlow.

Problem

Serving JumpStart Foundation Models incurs performance and cost challenges.

It is challenging to serve JumpStart Foundation Models efficiently in Amazon Sagemaker. The models are computationally heavy, incurring high costs and performance problems.

Solution

PeriFlow has been integrated with Amazon Sagemaker Jumpstart to serve JumpStart Foundation Models.

PeriFlow can be used with NCSOFT VARCO LLMs. Users of VARCO LLMs enjoy high speed and low cost of serving LLMs.

Result

Harness the power of PeriFlow to serve JumpStart Foundation Models

Users can effortlessly utilize NCSOFT VARCO LLMs on PeriFlow, resulting in cost reduction within Amazon Sagemaker Jumpstart.

usecases_04

TUNiB’s emotional chatbots with PeriFlow cloud

Launch diverse Generative AI models, managed by PeriFlow cloud

TUNiB's emotional chatbot services are earning accolades with PeriFlow cloud - FriendliAI's managed service for serving LLM.

Problem

Managing multiple AI models incurs significant time and costs

The client required to oversee the deployment of various Generative AI models to manage unpredictable real-time requests.

Solution

Use PeriFlow cloud for various models

PeriFlow cloud has enabled TUNiB to handle real-time executions with ease while also managing the number of deployments necessary to minimize operation costs. In addition, PeriFlow has further improved its model deployments, significantly reducing both costs and latency.

Result

Convenience and dependable service without the need for self-management

With PeriFlow cloud, TUNiB is able to deliver enhanced performance in interactive and creative AI models, all while maintaining low operating costs and request latency.

usecases_03

Lee Luda (이루다) 2.0 blooms with PeriFlow

Realize efficient Generative AI with PeriFlow

Scatter Lab's renewed chatbot service is accepting praise with PeriFlow, FriendliAI's LLM serving engine that speeds up Generative AI.

Problem

Quality and size of generative model comes with its own cost

The client company wanted their model to produce real-time responses based on current context, which required 17 times more parameters than the original version.

Solution

Use PeriFlow for Lee Luda 2.0

Scatter Lab adopted PeriFlow to serve their model. PeriFlow was able to handle the real-time executions while reducing the cost and the latency dramatically.

Result

Reliable service with much improved efficiency

With PeriFlow, Lee Luda 2.0 had launched successfully and is being used in practice. Its enhanced performance of interactive and creative communication is accepting praises while maintaining the cost and latency of the service.

usecases_02

Training a Large Language Model (LLM) with PeriFlow training

Swift and Sound; develop your own large-scale AI with PeriFlow training

We developed a GPT-3 13B model to show what it's like to train a LLM on PeriFlow training.

Problem

Too much cost for large-scale AI training

Normally, training a large-scale model takes a lot of resources. If you take distributed learning, the burden of faults and loads would only increase.

Solution

Automated and optimized training experience

On PeriFlow, we could enjoy its special support for distributed learning along with various optimization techniques. PeriFlow also handled the errors and performance problems to ensure sound training.

Result

Made large-scale AI simple

Manipulating PeriFlow’s automatic matching for state-of-the-art training techniques, training a 13 billion parameter model was felt like a breeze.

usecases_01

PeriFlow

Use PeriFlow for your generative language model

A client company using Transformer-based models was in a difficult situation, and we could help them with our efficient serving system.

Problem

Resource-consuming generative models

Transformer-based generative models take massive cost for serving, making companies reluctant to use them.

Solution

Tens of times increased efficiency using PeriFlow

We offered PeriFlow, our new serving system for generative models. With PeriFlow, the generative model of the company was able to experience much improves throughput with low and steady latency.

Result

More affordable use of generative models

The company can use the models without cost concerns. PeriFlow can be helpful to any kind of generative models in need of high performance with low cost.

We use cookiesWe use cookies to enhance your browsing experience on our website. By clicking “Accept all,” you consent to our use of cookies.
scroll to top