GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Jeff98
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Jeff98
Model Tree
openbmb/MiniCPM5-2B
Input Modalities
Output Modalities
Supported Functionality
openbmb/MiniCPM5-2BThis repository is intended primarily for inference and deployment. It does not modify the model architecture or provide additional fine-tuning.
The repository contains the OpenVINO model and the files required for LLM serving, including:
text
openvino_model.xmlopenvino_model.binopenvino_tokenizer.xmlopenvino_tokenizer.binopenvino_detokenizer.xmlopenvino_detokenizer.bintokenizer_config.jsonchat_template.jinja
The OpenVINO tokenizer and detokenizer are provided so that the model can be used directly with OpenVINO GenAI and OpenVINO Model Server.
The chat_template.jinja file is inherited from the original MiniCPM5 model and is required by OVMS when using the OpenAI-compatible Chat Completions API.
The model can be served using OVMS.
Example:
bash
ovms \--model_name MiniCPM5-2B \--model_path /path/to/MiniCPM5-2B-openvino \--rest_port 8000 \--cache_size 0 \--target_device GPU
On Windows PowerShell:
powershell
ovms `--model_name MiniCPM5-2B `--model_path "C:\path\to\MiniCPM5-2B-openvino" `--rest_port 8000 `--cache_size 0 `--target_device GPU
For CPU inference, use:
text
--target_device CPU
The available OpenVINO devices depend on your local hardware and OpenVINO installation.
Once OVMS is running, the model can be accessed through its OpenAI-compatible REST API.
bash
curl http://localhost:8000/v3/chat/completions \-H "Content-Type: application/json" \-d '{"model": "MiniCPM5-2B","messages": [{"role": "user","content": "Explain what OpenVINO is in simple terms."}],"max_tokens": 256}'
The chat endpoint uses the original MiniCPM5 chat template included in this repository.
python
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v3",api_key="unused")response = client.chat.completions.create(model="MiniCPM5-2B",messages=[{"role": "user","content": "Write a Python function to compute Fibonacci numbers."}],max_tokens=256,)print(response.choices[0].message.content)
This model was converted from:
text
openbmb/MiniCPM5-2B
to OpenVINO IR format.
The conversion preserves the original model behavior as closely as possible while enabling inference through the OpenVINO runtime.
The tokenizer and detokenizer are additionally converted to OpenVINO IR so that the model can be served directly through OVMS without relying on a separate Python tokenizer during inference.
Conceptually, the deployment package consists of:
text
Original MiniCPM5-2B│├── Language Model│ ↓│ OpenVINO Model IR│├── Tokenizer│ ↓│ OpenVINO Tokenizer IR│├── Detokenizer│ ↓│ OpenVINO Detokenizer IR│└── Chat Template↓chat_template.jinja
This repository is suitable for:
For model capabilities, supported tasks, training details, and evaluation results, please refer to the original model:
This repository does not introduce any additional model training or alignment beyond the original MiniCPM5-2B model.
Therefore, the model inherits the limitations of the original model, including possible:
The OpenVINO conversion may also introduce small numerical differences compared with inference using the original framework.
Performance and memory consumption depend strongly on:
This repository is intended for experimentation and inference rather than as a guarantee of identical numerical output across all backends.
MiniCPM5-2B can generate and complete source code. However, this repository is not specifically fine-tuned as an IDE Fill-in-the-Middle (FIM) autocomplete model.
For latency-sensitive IDE autocomplete workloads, dedicated code-completion models may provide better completion quality.
MiniCPM5-2B may still be useful for:
All model architecture, training, tokenizer design, and model capabilities originate from the MiniCPM5 project by OpenBMB.
Original model:
This repository only provides the OpenVINO conversion and deployment artifacts.
Please follow the license and usage requirements of the original model.