Browse generative AI models
supported by Friendli Engine
Friendli EngineModel library
Model architecture | Models | Hugging Face Hub repos (example) | LoRA |
---|---|---|---|
ArcticForCausalLM | Arctic | ||
BaichuanForCausalLM | Baichuan | ||
BlenderbotForConditionalGeneration | Blenderbot | ||
BloomForCausalLM | BLOOM
BLOOMZ | bigscience/bloom , etc. | |
CohereForCausalLM | Command R
Command R+ | ||
DbrxForCausalLM | DBRX | databricks/dbrx-instruct , etc. | |
DeepseekForCausalLM | DeepSeek | ||
ExaoneForCausalLM | EXAONE | ||
FalconForCausalLM | Falcon | tiiuae/falcon-40b , etc. | |
Gemma2ForCausalLM | Gemma 2 | ||
GemmaForCausalLM | CodeGemma
Gemma | ||
GPT2LMHeadModel | GPT2 | openai-community/gpt2-xl , etc. | |
GPTJForCausalLM | GPT-J | EleutherAI/gpt-j-6b , etc. | |
GPTNeoXForCausalLM | GPT-NeoX
Pythia
Dolly
StableLM | ||
Grok1ForCausalLM | Grok-1 | ||
LlamaForCausalLM | Llama 3.1
Llama 3
Llama 2
CodeLlama
OpenLLaMA
Hermes 3
Vicuna
Yi
WizardLM
WizardMath
WizardCoder | ||
MistralForCausalLM | Mistral 7B
Mistral Nemo
Mistral Large 2
Mathstral | ||
MixtralForCausalLM | Mixtral 8x7B
Mixtral 8x22B
Zephyr | ||
MPTForCausalLM | MPT | mosaicml/mpt-30b , etc. | |
MT5ForConditionalGeneration | MT5 | google/mt5-xxl , etc. | |
OPTForcausalLM | OPT | facebook/opt-66b , etc. | |
Phi3ForCausalLM | Phi-3.5
Phi-3 | ||
PhiForCausalLM | Phi-2
Phi-1 | microsoft/phi-2 , etc. | |
Qwen2ForCausalLM | Qwen2.5
Qwen2
Qwen1.5 | ||
SolarForCausalLM | Solar | ||
Starcoder2ForCausalLM | StarCoder 2 | bigcode/starcoder2-15b , etc. | |
T5ForConditionalGeneration | FLAN-T5 | google/flan-t5-xxl , etc. |
Friendli Engine supports a wide array of quantization techniques, including FP8, INT8, and AWQ in all models.
The list above may not exhaustive. If your model does not belong to one of the above models, please check our documentation for more information or contact us for support.
HOW TO USE
Three ways to run generative AI models with Friendli Engine:
02
Friendli Container
Serve LLM and LMM inferences with Friendli Engine in your GPU environment
Learn more03
Friendli Serverless Endpoints
Call our fast and affordable API for open-source generative AI models
Learn more