ministral3_8b_w4a16_unsloth API & Inference Endpoint

Key Features

Ministral 3 8B consists of two main architectural components:

8.4B Language Model
0.4B Vision Encoder

The Ministral 3 8B Instruct model offers the following capabilities:

Vision: Enables the model to analyze images and provide insights based on visual content, in addition to text.
Multilingual: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic.
System Prompt: Maintains strong adherence and support for system prompts.
Agentic: Offers best-in-class agentic capabilities with native function calling and JSON outputting.
Edge-Optimized: Delivers best-in-class performance at a small scale, deployable anywhere.
Apache 2.0 License: Open-source license allowing usage and modification for both commercial and non-commercial purposes.
Large Context Window: Supports a 256k context window.

Use Cases

Perfect for balanced performance in local or embedded systems, combining versatility with efficiency.

Chat interfaces in constrained environments
Local daily-driver AI assistant
Image/document description and understanding
Translation and content generation
Specialized agentic use cases
Fine-tuning and specialization
And more...

Bringing advanced AI capabilities to resource-constrained environments.

Ministral 3 Family

Table with columns: Model Name, Type, Precision, Link
Model Name	Type	Precision	Link
Ministral 3 3B Base 2512	Base pre-trained	BF16	Hugging Face
Ministral 3 3B Instruct 2512	Instruct post-trained	FP8	Hugging Face
Ministral 3 3B Reasoning 2512	Reasoning capable	BF16	Hugging Face
Ministral 3 8B Base 2512	Base pre-trained	BF16	Hugging Face
Ministral 3 8B Instruct 2512	Instruct post-trained	FP8	Hugging Face
Ministral 3 8B Reasoning 2512	Reasoning capable	BF16	Hugging Face
Ministral 3 14B Base 2512	Base pre-trained**	BF16	Hugging Face
Ministral 3 14B Instruct 2512	Instruct post-trained	FP8	Hugging Face
Ministral 3 14B Reasoning 2512	Reasoning capable	BF16	Hugging Face

Other formats available here.

Benchmark Results

We compare Ministral 3 to similar sized models.

Reasoning

Table with columns: Model, AIME25, AIME24, GPQA Diamond, LiveCodeBench
Model	AIME25	AIME24	GPQA Diamond	LiveCodeBench
Ministral 3 14B	0.850	0.898	0.712	0.646
Qwen3-14B (Thinking)	0.737	0.837	0.663	0.593

Ministral 3 8B	0.787	0.860	0.668	0.616
Qwen3-VL-8B-Thinking	0.798	0.860	0.671	0.580

Ministral 3 3B	0.721	0.775	0.534	0.548
Qwen3-VL-4B-Thinking	0.697	0.729	0.601	0.513

Instruct

Table with columns: Model, Arena Hard, WildBench, MATH Maj@1, MM MTBench
Model	Arena Hard	WildBench	MATH Maj@1	MM MTBench
Ministral 3 14B	0.551	68.5	0.904	8.49
Qwen3 14B (Non-Thinking)	0.427	65.1	0.870	NOT MULTIMODAL
Gemma3-12B-Instruct	0.436	63.2	0.854	6.70

Ministral 3 8B	0.509	66.8	0.876	8.08
Qwen3-VL-8B-Instruct	0.528	66.3	0.946	8.00

Ministral 3 3B	0.305	56.8	0.830	7.83
Qwen3-VL-4B-Instruct	0.438	56.8	0.900	8.01
Qwen3-VL-2B-Instruct	0.163	42.2	0.786	6.36
Gemma3-4B-Instruct	0.318	49.1	0.759	5.23

Base

Table with columns: Model, Multilingual MMLU, MATH CoT 2-Shot, AGIEval 5-shot, MMLU Redux 5-shot, MMLU 5-shot, TriviaQA 5-shot
Model	Multilingual MMLU	MATH CoT 2-Shot	AGIEval 5-shot	MMLU Redux 5-shot	MMLU 5-shot	TriviaQA 5-shot
Ministral 3 14B	0.742	0.676	0.648	0.820	0.794	0.749
Qwen3 14B Base	0.754	0.620	0.661	0.837	0.804	0.703
Gemma 3 12B Base	0.690	0.487	0.587	0.766	0.745	0.788

Ministral 3 8B	0.706	0.626	0.591	0.793	0.761	0.681
Qwen 3 8B Base	0.700	0.576	0.596	0.794	0.760	0.639

Ministral 3 3B	0.652	0.601	0.511	0.735	0.707	0.592
Qwen 3 4B Base	0.677	0.405	0.570	0.759	0.713	0.530
Gemma 3 4B Base	0.516	0.294	0.430	0.626	0.589	0.640

Usage

The model can be used with the following frameworks;

vllm: See here
transformers: See here

vLLM

We recommend using this model with vLLM.

Installation

Make sure to install vLLM >= 0.12.0:

text

pip install vllm --upgrade

Doing so should automatically install mistral_common >= 1.8.6.

To check:

text

python -c "import mistral_common; print(mistral_common.__version__)"

You can also make use of a ready-to-go docker image or on the docker hub.

Serve

Due to their size and the FP8 format of their weights Ministral-3-3B-Instruct-2512, Ministral-3-8B-Instruct-2512 and Ministral-3-14B-Instruct-2512 can run on a single 1xH200 GPU.

A simple launch command is:

bash

vllm serve mistralai/Ministral-3-8B-Instruct-2512 \  --enable-auto-tool-choice --tool-call-parser mistral

Key parameter notes:

enable-auto-tool-choice: Required when enabling tool usage.
tool-call-parser mistral: Required when enabling tool usage.

Additional flags:

You can set --max-model-len to preserve memory. By default it is set to 262144 which is quite large but not necessary for most scenarios.
You can set --max-num-batched-tokens to balance throughput and latency, higher means higher throughput but higher latency.

Usage of the model

Here we asumme that the model mistralai/Ministral-3-8B-Instruct-2512 is served and you can ping it to the domain localhost with the port 8000 which is the default for vLLM.

Let's see if the Ministral 3 knows when to pick a fight !

python

from datetime import datetime, timedelta from openai import OpenAIfrom huggingface_hub import hf_hub_download # Modify OpenAI's API key and API base to use vLLM's API server.openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1" TEMP = 0.15MAX_TOK = 262144 client = OpenAI(    api_key=openai_api_key,    base_url=openai_api_base,) models = client.models.list()model = models.data[0].id  def load_system_prompt(repo_id: str, filename: str) -> str:    file_path = hf_hub_download(repo_id=repo_id, filename=filename)    with open(file_path, "r") as file:        system_prompt = file.read()    today = datetime.today().strftime("%Y-%m-%d")    yesterday = (datetime.today() - timedelta(days=1)).strftime("%Y-%m-%d")    model_name = repo_id.split("/")[-1]    return system_prompt.format(name=model_name, today=today, yesterday=yesterday)  SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438" messages = [    {"role": "system", "content": SYSTEM_PROMPT},    {        "role": "user",        "content": [            {                "type": "text",                "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.",            },            {"type": "image_url", "image_url": {"url": image_url}},        ],    },] print(messages)  response = client.chat.completions.create(    model=model,    messages=messages,    temperature=TEMP,    max_tokens=MAX_TOK,) print(response.choices[0].message.content)

Let's solve some equations thanks to our simple Python calculator tool.

python

import jsonfrom openai import OpenAIfrom huggingface_hub import hf_hub_download # Modify OpenAI's API key and API base to use vLLM's API server.openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1" TEMP = 0.15MAX_TOK = 262144 client = OpenAI(    api_key=openai_api_key,    base_url=openai_api_base,) models = client.models.list()model = models.data[0].id  def load_system_prompt(repo_id: str, filename: str) -> str:    file_path = hf_hub_download(repo_id=repo_id, filename=filename)    with open(file_path, "r") as file:        system_prompt = file.read()    return system_prompt  SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt") image_url = "https://math-coaching.com/img/fiche/46/expressions-mathematiques.jpg"  def my_calculator(expression: str) -> str:    return str(eval(expression))  tools = [    {        "type": "function",        "function": {            "name": "my_calculator",            "description": "A calculator that can evaluate a mathematical expression.",            "parameters": {                "type": "object",                "properties": {                    "expression": {                        "type": "string",                        "description": "The mathematical expression to evaluate.",                    },                },                "required": ["expression"],            },        },    },    {        "type": "function",        "function": {            "name": "rewrite",            "description": "Rewrite a given text for improved clarity",            "parameters": {                "type": "object",                "properties": {                    "text": {                        "type": "string",                        "description": "The input text to rewrite",                    }                },            },        },    },] messages = [    {"role": "system", "content": SYSTEM_PROMPT},    {        "role": "user",        "content": [            {                "type": "text",                "text": "Thanks to your calculator, compute the results for the equations that involve numbers displayed in the image.",            },            {                "type": "image_url",                "image_url": {                    "url": image_url,                },            },        ],    },] response = client.chat.completions.create(    model=model,    messages=messages,    temperature=TEMP,    max_tokens=MAX_TOK,    tools=tools,    tool_choice="auto",) tool_calls = response.choices[0].message.tool_calls results = []for tool_call in tool_calls:    function_name = tool_call.function.name    function_args = tool_call.function.arguments    if function_name == "my_calculator":        result = my_calculator(**json.loads(function_args))        results.append(result) messages.append({"role": "assistant", "tool_calls": tool_calls})for tool_call, result in zip(tool_calls, results):    messages.append(        {            "role": "tool",            "tool_call_id": tool_call.id,            "name": tool_call.function.name,            "content": result,        }    )  response = client.chat.completions.create(    model=model,    messages=messages,    temperature=TEMP,    max_tokens=MAX_TOK,) print(response.choices[0].message.content)

Ministral 3 can follow your instructions to the letter.

python

from openai import OpenAIfrom huggingface_hub import hf_hub_download # Modify OpenAI's API key and API base to use vLLM's API server.openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1" TEMP = 0.15MAX_TOK = 262144 client = OpenAI(    api_key=openai_api_key,    base_url=openai_api_base,) models = client.models.list()model = models.data[0].id  def load_system_prompt(repo_id: str, filename: str) -> str:    file_path = hf_hub_download(repo_id=repo_id, filename=filename)    with open(file_path, "r") as file:        system_prompt = file.read()    return system_prompt  SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt") messages = [    {"role": "system", "content": SYSTEM_PROMPT},    {        "role": "user",        "content": "Write me a sentence where every word starts with the next letter in the alphabet - start with 'a' and end with 'z'.",    },] response = client.chat.completions.create(    model=model,    messages=messages,    temperature=TEMP,    max_tokens=MAX_TOK,) assistant_message = response.choices[0].message.contentprint(assistant_message)

Transformers

You can also use Ministral 3 8B Instruct 2512 with Transformers !

Transformers very recently added prelimenary support for FP8, so please make sure to install from main:

uv pip install git+https://github.com/huggingface/transformers

To make the best use of our model with Transformers make sure to have installed mistral-common >= 1.8.6 to use our tokenizer.

bash

pip install mistral-common --upgrade

Try it out by running the following snippet.

[!Tip] By default Transformers will load the checkpoint in FP8 and dequantize it to BF16 on the fly, which means the model currently does not make use of accelerated FP8-kernels. Compatibility with accelerated FP8-kernels is currently worked on and will be available in a couple of weeks. Stay tuned!

Then load our tokenizer along with the model and generate:

python

import torchfrom transformers import Mistral3ForConditionalGeneration, MistralCommonBackend model_id = "mistralai/Ministral-3-8B-Instruct-2512" tokenizer = MistralCommonBackend.from_pretrained(model_id)model = Mistral3ForConditionalGeneration.from_pretrained(model_id, device_map="auto") image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438" messages = [    {        "role": "user",        "content": [            {                "type": "text",                "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.",            },            {"type": "image_url", "image_url": {"url": image_url}},        ],    },] tokenized = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True) tokenized["input_ids"] = tokenized["input_ids"].to(device="cuda")tokenized["pixel_values"] = tokenized["pixel_values"].to(dtype=torch.bfloat16, device="cuda")image_sizes = [tokenized["pixel_values"].shape[-2:]] output = model.generate(    **tokenized,    image_sizes=image_sizes,    max_new_tokens=512,)[0] decoded_output = tokenizer.decode(output[len(tokenized["input_ids"][0]):])print(decoded_output)

Note:

Transformers allows you to automatically convert the checkpoint to Bfloat16. To so simple load the model as follows:

from transformers import Mistral3ForConditionalGeneration, FineGrainedFP8Config model_id = "mistralai/Ministral-3-8B-Instruct-2512"model = Mistral3ForConditionalGeneration.from_pretrained(    model_id,    device_map="auto",    quantization_config=FineGrainedFP8Config(dequantize=True))

License

This model is licensed under the Apache 2.0 License.

You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.

Model Name

Type

Precision

Link

Ministral 3 3B Base 2512

Base pre-trained

BF16

Hugging Face

Ministral 3 3B Instruct 2512

Instruct post-trained

FP8

Hugging Face

Ministral 3 3B Reasoning 2512

Reasoning capable

BF16

Hugging Face

Ministral 3 8B Base 2512

Base pre-trained

BF16

Hugging Face

Ministral 3 8B Instruct 2512

Instruct post-trained

FP8

Hugging Face

Ministral 3 8B Reasoning 2512

Reasoning capable

BF16

Hugging Face

Ministral 3 14B Base 2512

Base pre-trained**

BF16

Hugging Face

Ministral 3 14B Instruct 2512

Instruct post-trained

FP8

Hugging Face

Ministral 3 14B Reasoning 2512

Reasoning capable

BF16

Hugging Face

Model

AIME25

AIME24

GPQA Diamond

LiveCodeBench

Ministral 3 14B

0.850

0.898

0.712

0.646

Qwen3-14B (Thinking)

0.737

0.837

0.663

0.593

Ministral 3 8B

0.787

0.860

0.668

0.616

Qwen3-VL-8B-Thinking

0.798

0.860

0.671

0.580

Ministral 3 3B

0.721

0.775

0.534

0.548

Qwen3-VL-4B-Thinking

0.697

0.729

0.601

0.513

Model

Arena Hard

WildBench

MATH Maj@1

MM MTBench

Ministral 3 14B

0.551

68.5

0.904

8.49

Qwen3 14B (Non-Thinking)

0.427

65.1

0.870

NOT MULTIMODAL

Gemma3-12B-Instruct

0.436

63.2

0.854

6.70

Ministral 3 8B

0.509

66.8

0.876

8.08

Qwen3-VL-8B-Instruct

0.528

66.3

0.946

8.00

Ministral 3 3B

0.305

56.8

0.830

7.83

Qwen3-VL-4B-Instruct

0.438

56.8

0.900

8.01

Qwen3-VL-2B-Instruct

0.163

42.2

0.786

6.36

Gemma3-4B-Instruct

0.318

49.1

0.759

5.23

Model

Multilingual MMLU

MATH CoT 2-Shot

AGIEval 5-shot

MMLU Redux 5-shot

MMLU 5-shot

TriviaQA 5-shot

Ministral 3 14B

0.742

0.676

0.648

0.820

0.794

0.749

Qwen3 14B Base

0.754

0.620

0.661

0.837

0.804

0.703

Gemma 3 12B Base

0.690

0.487

0.587

0.766

0.745

0.788

Ministral 3 8B

0.706

0.626

0.591

0.793

0.761

0.681

Qwen 3 8B Base

0.700

0.576

0.596

0.794

0.760

0.639

Ministral 3 3B

0.652

0.601

0.511

0.735

0.707

0.592

Qwen 3 4B Base

0.677

0.405

0.570

0.759

0.713

0.530

Gemma 3 4B Base

0.516

0.294

0.430

0.626

0.589

0.640

from datetime import datetime, timedelta from openai import OpenAIfrom huggingface_hub import hf_hub_download # Modify OpenAI's API key and API base to use vLLM's API server.openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1" TEMP = 0.15MAX_TOK = 262144 client = OpenAI( api_key=openai_api_key, base_url=openai_api_base,) models = client.models.list()model = models.data[0].id def load_system_prompt(repo_id: str, filename: str) -> str: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() today = datetime.today().strftime("%Y-%m-%d") yesterday = (datetime.today() - timedelta(days=1)).strftime("%Y-%m-%d") model_name = repo_id.split("/")[-1] return system_prompt.format(name=model_name, today=today, yesterday=yesterday) SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438" messages = [ {"role": "system", "content": SYSTEM_PROMPT}, { "role": "user", "content": [ { "type": "text", "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.", }, {"type": "image_url", "image_url": {"url": image_url}}, ], },] print(messages) response = client.chat.completions.create( model=model, messages=messages, temperature=TEMP, max_tokens=MAX_TOK,) print(response.choices[0].message.content)

import jsonfrom openai import OpenAIfrom huggingface_hub import hf_hub_download # Modify OpenAI's API key and API base to use vLLM's API server.openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1" TEMP = 0.15MAX_TOK = 262144 client = OpenAI( api_key=openai_api_key, base_url=openai_api_base,) models = client.models.list()model = models.data[0].id def load_system_prompt(repo_id: str, filename: str) -> str: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() return system_prompt SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt") image_url = "https://math-coaching.com/img/fiche/46/expressions-mathematiques.jpg" def my_calculator(expression: str) -> str: return str(eval(expression)) tools = [ { "type": "function", "function": { "name": "my_calculator", "description": "A calculator that can evaluate a mathematical expression.", "parameters": { "type": "object", "properties": { "expression": { "type": "string", "description": "The mathematical expression to evaluate.", }, }, "required": ["expression"], }, }, }, { "type": "function", "function": { "name": "rewrite", "description": "Rewrite a given text for improved clarity", "parameters": { "type": "object", "properties": { "text": { "type": "string", "description": "The input text to rewrite", } }, }, }, },] messages = [ {"role": "system", "content": SYSTEM_PROMPT}, { "role": "user", "content": [ { "type": "text", "text": "Thanks to your calculator, compute the results for the equations that involve numbers displayed in the image.", }, { "type": "image_url", "image_url": { "url": image_url, }, }, ], },] response = client.chat.completions.create( model=model, messages=messages, temperature=TEMP, max_tokens=MAX_TOK, tools=tools, tool_choice="auto",) tool_calls = response.choices[0].message.tool_calls results = []for tool_call in tool_calls: function_name = tool_call.function.name function_args = tool_call.function.arguments if function_name == "my_calculator": result = my_calculator(**json.loads(function_args)) results.append(result) messages.append({"role": "assistant", "tool_calls": tool_calls})for tool_call, result in zip(tool_calls, results): messages.append( { "role": "tool", "tool_call_id": tool_call.id, "name": tool_call.function.name, "content": result, } ) response = client.chat.completions.create( model=model, messages=messages, temperature=TEMP, max_tokens=MAX_TOK,) print(response.choices[0].message.content)

from openai import OpenAIfrom huggingface_hub import hf_hub_download # Modify OpenAI's API key and API base to use vLLM's API server.openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1" TEMP = 0.15MAX_TOK = 262144 client = OpenAI( api_key=openai_api_key, base_url=openai_api_base,) models = client.models.list()model = models.data[0].id def load_system_prompt(repo_id: str, filename: str) -> str: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() return system_prompt SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt") messages = [ {"role": "system", "content": SYSTEM_PROMPT}, { "role": "user", "content": "Write me a sentence where every word starts with the next letter in the alphabet - start with 'a' and end with 'z'.", },] response = client.chat.completions.create( model=model, messages=messages, temperature=TEMP, max_tokens=MAX_TOK,) assistant_message = response.choices[0].message.contentprint(assistant_message)

import torchfrom transformers import Mistral3ForConditionalGeneration, MistralCommonBackend model_id = "mistralai/Ministral-3-8B-Instruct-2512" tokenizer = MistralCommonBackend.from_pretrained(model_id)model = Mistral3ForConditionalGeneration.from_pretrained(model_id, device_map="auto") image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438" messages = [ { "role": "user", "content": [ { "type": "text", "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.", }, {"type": "image_url", "image_url": {"url": image_url}}, ], },] tokenized = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True) tokenized["input_ids"] = tokenized["input_ids"].to(device="cuda")tokenized["pixel_values"] = tokenized["pixel_values"].to(dtype=torch.bfloat16, device="cuda")image_sizes = [tokenized["pixel_values"].shape[-2:]] output = model.generate( **tokenized, image_sizes=image_sizes, max_new_tokens=512,)[0] decoded_output = tokenizer.decode(output[len(tokenized["input_ids"][0]):])print(decoded_output)

from transformers import Mistral3ForConditionalGeneration, FineGrainedFP8Config model_id = "mistralai/Ministral-3-8B-Instruct-2512"model = Mistral3ForConditionalGeneration.from_pretrained( model_id, device_map="auto", quantization_config=FineGrainedFP8Config(dequantize=True))

ministral3_8b_w4a16_unsloth

README

Key Features

Use Cases

Ministral 3 Family

Benchmark Results

Reasoning

Instruct

Base

Usage

vLLM

Installation

Serve

Usage of the model

Transformers

License

Explore FriendliAI today

README

Key Features

Use Cases

Ministral 3 Family

Benchmark Results

Reasoning

Instruct

Base

Usage

vLLM

Installation

Serve

Usage of the model

Transformers

License