Browse generative AI models
supported by Friendli Engine
Friendli EngineModel library
Model architecture | Models | Hugging Face Hub repos (example) | LoRA |
---|---|---|---|
ArcticForCausalLM | Arctic | ||
BaichuanForCausalLM | Baichuan | ||
BlenderbotForConditionalGeneration | Blenderbot | ||
BloomForCausalLM | BLOOM
BLOOMZ | bigscience/bloom , etc. | |
CohereForCausalLM | Command R
Command R+ | ||
DbrxForCausalLM | DBRX | databricks/dbrx-instruct , etc. | |
DeepseekForCausalLM | DeepSeek | ||
ExaoneForCausalLM | EXAONE | ||
FalconForCausalLM | Falcon | tiiuae/falcon-40b , etc. | |
Gemma2ForCausalLM | Gemma 2 | ||
GemmaForCausalLM | CodeGemma
Gemma | ||
GPT2LMHeadModel | GPT2 | openai-community/gpt2-xl , etc. | |
GPTBigCodeForCausalLM | StarCoder
SantaCoder | bigcode/starcoder , etc. | |
GPTJForCausalLM | GPT-J | EleutherAI/gpt-j-6b , etc. | |
GPTNeoXForCausalLM | GPT-NeoX
Pythia
Dolly
StableLM | ||
Grok1ForCasualLM | Grok-1 | ||
LlamaForCausalLM | Llama 3.1
Llama 3
Llama 2
CodeLlama
OpenLLaMA
Vicuna
Yi
WizardLM
WizardMath
WizardCoder | ||
MistralForCausalLM | Mistral 7B
Mistral Nemo
Mistral Large 2
Mathstral | ||
MixtralForCausalLM | Mixtral 8x7B
Mixtral 8x22B
Zephyr | ||
MPTForCausalLM | MPT | mosaicml/mpt-30b , etc. | |
MT5ForConditionalGeneration | MT5 | google/mt5-xxl , etc. | |
OPTForcausalLM | OPT | facebook/opt-66b , etc. | |
Phi3ForCausalLM | Phi-3.5
Phi-3 | ||
PhiForCausalLM | Phi-1
Phi-2 | microsoft/phi-2 , etc. | |
Qwen2ForCausalLM | Qwen1.5
Qwen2 | ||
SolarForCausalLM | Solar | ||
Starcoder2ForCausalLM | StarCoder 2 | bigcode/starcoder2-15b , etc. | |
T5ForConditionalGeneration | FLAN-T5 | google/flan-t5-xxl , etc. |
Friendli Engine supports a wide array of quantization techniques, including FP8, INT8, and AWQ in all models.
The list above may not exhaustive. If your model does not belong to one of the above models, please check our documentation for more information or contact us for support.
HOW TO USE
Three ways to run generative AI models with Friendli Engine:
02
Friendli Container
Serve LLMs/LMMs inferences with Friendli Engine in your GPU environment
Learn more03
Friendli Serverless Endpoints
Call fast and affordable API for open-source generative AI models
Learn more