GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
mistralai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
mistralai
Model Tree
Input Modalities
Output Modalities
Supported Functionality
When available, we report numbers previously published by other model providers, otherwise we re-evaluate them using our own evaluation harness.
| Model | MMLU (5-shot) | MMLU Pro (5-shot CoT) | TriviaQA | GPQA Main (5-shot CoT) | MMMU |
|---|---|---|---|---|---|
| Small 3.1 24B Base | 81.01% | 56.03% | 80.50% | 37.50% | 59.27% |
| Gemma 3 27B PT | 78.60% | 52.20% | 81.30% | 24.30% | 56.10% |
We recommend using Mistral-Small 3.1 Base with the vLLM library. Note however that this is a pretrained-only checkpoint and thus not ready to work as an instruction model out-of-the-box. For a production-ready instruction model please use Mistral-Small-3.1-24B-Instruct-2503.
Installation
We recommend using this model with the vLLM library to implement production-ready inference pipelines.
Make sure you install vLLM >= 0.8.1:
markdown
pip install vllm --ugrade
Doing so should automatically install mistral_common >= 1.5.4.
To check:
markdown
python -c "import mistral_common; print(mistral_common.__version__)"
You can also make use of a ready-to-go docker image or on the docker hub.
Example
py
from vllm import LLMfrom vllm.sampling_params import SamplingParamsfrom vllm.inputs.data import TokensPromptimport requestsfrom PIL import Imagefrom io import BytesIOfrom vllm.multimodal import MultiModalDataBuiltinsfrom mistral_common.protocol.instruct.messages import TextChunk, ImageURLChunkmodel_name = "mistralai/Mistral-Small-3.1-24B-Base-2503"sampling_params = SamplingParams(max_tokens=8192)llm = LLM(model=model_name, tokenizer_mode="mistral")url = "https://huggingface.co/datasets/patrickvonplaten/random_img/resolve/main/yosemite.png"response = requests.get(url)image = Image.open(BytesIO(response.content))prompt = "The image shows a"user_content = [ImageURLChunk(image_url=url), TextChunk(text=prompt)]tokenizer = llm.llm_engine.tokenizer.tokenizer.mistral.instruct_tokenizertokens, _ = tokenizer.encode_user_content(user_content, False)prompt = TokensPrompt(prompt_token_ids=tokens, multi_modal_data=MultiModalDataBuiltins(image=[image]))outputs = llm.generate(prompt, sampling_params=sampling_params)print(outputs[0].outputs[0].text)# ' scene in Yosemite Valley and was taken at ISO 250 with an aperture of f/16 and a shutter speed of 1/18 second. ...'
Transformers-compatible model weights are also uploaded (thanks a lot @cyrilvallez). However the transformers implementation was not throughly tested, but only on "vibe-checks". Hence, we can only ensure 100% correct behavior when using the original weight format with vllm (see above).