Key Features

Ministral 3 8B consists of two main architectural components:

• 8.4B Language Model
• 0.4B Vision Encoder

The Ministral 3 8B Base model offers the following capabilities:

• Vision: Enables the model to analyze images and provide insights based on visual content, in addition to text.
• Multilingual: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic.
• Edge-Optimized: Delivers best-in-class performance at a small scale, deployable anywhere.
• Apache 2.0 License: Open-source license allowing usage and modification for both commercial and non-commercial purposes.
• Large Context Window: Supports a 256k context window.

Use Cases

Perfect for balanced performance in local or embedded systems, combining versatility with efficiency.

• Chat interfaces in constrained environments
• Local daily-driver AI assistant
• Image/document description and understanding
• Translation and content generation
• Specialized agentic use cases
• Fine-tuning and specialization
• And more...

Bringing advanced AI capabilities to resource-constrained environments.

Ministral 3 Family

Other formats available here.

Benchmark Results

We compare Ministral 3 to similar sized models.

Reasoning

Instruct

Base

Usage

The model can be used with the following frameworks;

• vllm: See here
• transformers: See here

vLLM

We recommend using this model with vLLM.

Installation

Make sure to install vllm >= 1.12.0:

markdown
Copy code
pip install vllm --upgrade

Doing so should automatically install mistral_common >= 1.8.6.

To check:

markdown
Copy code
python -c "import mistral_common; print(mistral_common.__version__)"

You can also make use of a ready-to-go docker image or on the docker hub.

Serve

Due to their size and the BF16 format of their weights Ministral-3-3B-Base-2512 and Ministral-3-8B-Base-2512 can run on a single 1xH200 GPU.

A simple launch command is:

bash
Copy code
vllm serve mistralai/Ministral-3-8B-Instruct-2512 \
  --tokenizer_mode mistral --config_format mistral --load_format mistral

Additional flags:

• You can set --max-model-len to preserve memory. By default it is set to 262144 which is quite large but not necessary for most scenarios.
• You can set --max-num-batched-tokens to balance throughput and latency, higher means higher throughput but higher latency.

Usage of the model

Here we asumme that the model mistralai/Ministral-3-8B-Base-2512 is served and you can ping it to the domain localhost with the port 8000 which is the default for vLLM.

Quick test with the base model.

python
Copy code
from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"

TEMP = 0.15
MAX_TOK = 256

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

response = client.completions.create(
    model=model,
    prompt="What is the best thing in the universe ?",
    temperature=TEMP,
    max_tokens=MAX_TOK,
)

print(response.choices[0].text)

Transformers

You can also use Ministral 3 8B Base 2512 with Transformers ! Make sure to install Transformers from its first v5 release candidate or from "main":

markdown
Copy code
pip install transformers==5.0.0rc0

To make the best use of our model with Transformers make sure to have installed mistral-common >= 1.8.6 to use our tokenizer.

bash
Copy code
pip install mistral-common --upgrade

Then load our tokenizer along with the model and generate:

python
Copy code
from transformers import Mistral3ForConditionalGeneration, MistralCommonBackend, FineGrainedFP8Config

model_id = "mistralai/Ministral-3-8B-Base-2512"
model = Mistral3ForConditionalGeneration.from_pretrained(
    model_id,
    device_map="auto",
)
tokenizer = MistralCommonBackend.from_pretrained(model_id)

input_ids = tokenizer.encode("Once about a time, France was a", return_tensors="pt")
input_ids = input_ids.to("cuda")

output = model.generate(
    input_ids,
    max_new_tokens=30,
)[0]

decoded_output = tokenizer.decode(output[len(input_ids[0]):])
print(decoded_output)

License

This model is licensed under the Apache 2.0 License.

You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.