inference-optimization

Inkling-0.92B-MTP

README

License: apache-2.0

Load the model

This fixture has no published quantized variant.

python

import torchfrom transformers import InklingForConditionalGeneration MODEL_ID = "inference-optimization/Inkling-0.92B-MTP"model = InklingForConditionalGeneration.from_pretrained(    MODEL_ID, dtype=torch.bfloat16, attn_implementation="eager",)

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

inference-optimization

Model Tree

Base

this model

Input Modalities

TextAudioImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer