Ministral-3-14B-Base-2512

Key Features

Ministral 3 14B consists of two main architectural components:

• 13.5B Language Model
• 0.4B Vision Encoder

The Ministral 3 14B Base model offers the following capabilities:

• Vision: Enables the model to analyze images and provide insights based on visual content, in addition to text.
• Multilingual: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic.
• Edge-Optimized: Delivers best-in-class performance at a small scale, deployable anywhere.
• Apache 2.0 License: Open-source license allowing usage and modification for both commercial and non-commercial purposes.
• Large Context Window: Supports a 256k context window.

Use Cases

Private AI deployments where advanced capabilities meet practical hardware constraints:

• Private/custom chat and AI assistant deployments in constrained environments
• Advanced local agentic use cases
• Fine-tuning and specialization
• And more...

Bringing advanced AI capabilities to most environments.

Ministral 3 Family

Other formats available here.

Benchmark Results

We compare Ministral 3 to similar sized models.

Reasoning

Instruct

Base

Usage

The model can be used with the following frameworks;

• vllm: See here
• transformers: See here

vLLM

We recommend using this model with vLLM.

Installation

Make sure to install vllm >= 1.12.0:

markdown
Copy code
pip install vllm --upgrade

Doing so should automatically install mistral_common >= 1.8.6.

To check:

markdown
Copy code
python -c "import mistral_common; print(mistral_common.__version__)"

You can also make use of a ready-to-go docker image or on the docker hub.

Serve

To fully exploit the Ministral-3-14B-Base-2512 we recommed using 2xH200 GPUs for deployment due to its large context. However if you don't need a large context, you can fall back to a single GPU.

A simple launch command is:

bash
Copy code
vllm serve mistralai/Ministral-3-14B-Base-2512 --tensor-parallel-size 2 \
  --tokenizer_mode mistral --config_format mistral --load_format mistral

Additional flags:

• You can set --max-model-len to preserve memory. By default it is set to 262144 which is quite large but not necessary for most scenarios.
• You can set --max-num-batched-tokens to balance throughput and latency, higher means higher throughput but higher latency.

Usage of the model

Here we assume that the model mistralai/Ministral-3-14B-Base-2512 is served and you can ping it to the domain localhost with the port 8000 which is the default for vLLM.

Quick test with the base model.

python
Copy code
from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"

TEMP = 0.15
MAX_TOK = 256

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

response = client.completions.create(
    model=model,
    prompt="What is the best thing in the universe ?",
    temperature=TEMP,
    max_tokens=MAX_TOK,
)

print(response.choices[0].text)

Transformers

You can also use Ministral 3 14B Base 2512 with Transformers ! Make sure to install Transformers from its first v5 release candidate or from "main":

markdown
Copy code
pip install transformers==5.0.0rc0

To make the best use of our model with Transformers make sure to have installed mistral-common >= 1.8.6 to use our tokenizer.

bash
Copy code
pip install mistral-common --upgrade

Then load our tokenizer along with the model and generate:

python
Copy code
from transformers import Mistral3ForConditionalGeneration, MistralCommonBackend, FineGrainedFP8Config

model_id = "mistralai/Ministral-3-14B-Base-2512"
model = Mistral3ForConditionalGeneration.from_pretrained(
    model_id,
    device_map="auto",
)
tokenizer = MistralCommonBackend.from_pretrained(model_id)

input_ids = tokenizer.encode("Once about a time, France was a", return_tensors="pt")
input_ids = input_ids.to("cuda")

output = model.generate(
    input_ids,
    max_new_tokens=30,
)[0]

decoded_output = tokenizer.decode(output[len(input_ids[0]):])
print(decoded_output)

License

This model is licensed under the Apache 2.0 License.

You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.