GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
apolloparty
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
apolloparty
Model Tree
mistralai/Devstral-Small-2507
Input Modalities
Output Modalities
Supported Functionality
Devstral Small 1.1 achieves a score of 53.6% on SWE-Bench Verified, outperforming Devstral Small 1.0 by +6,8% and the second best state of the art model by +11.4%.
| Model | Agentic Scaffold | SWE-Bench Verified (%) |
|---|---|---|
| Devstral Small 1.1 | OpenHands Scaffold | 53.6 |
| Devstral Small 1.0 | OpenHands Scaffold | 46.8 |
| GPT-4.1-mini | OpenAI Scaffold | 23.6 |
| Claude 3.5 Haiku | Anthropic Scaffold | 40.6 |
| SWE-smith-LM 32B | SWE-agent Scaffold | 40.2 |
| Skywork SWE | OpenHands Scaffold | 38.0 |
| DeepSWE | R2E-Gym Scaffold | 42.2 |
When evaluated under the same test scaffold (OpenHands, provided by All Hands AI 🙌), Devstral exceeds far larger models such as Deepseek-V3-0324 and Qwen3 232B-A22B.
We recommend to use Devstral NVFP4A16 with the [vLLM >= 0.9.1](https://github.com/vllm-project/vllm/releases/tag/v0.9.1
Other methods are untested
We recommend using this model with the vLLM library to implement production-ready inference pipelines.
Installation
Make sure you install vLLM >= 0.9.1:
text
pip install vllm --extra-index-url https://download.pytorch.org/whl/cu128Also make sure to have installed mistral_common >= 1.7.0.
text
pip install mistral-common --upgradeTo check:
text
python -c "import mistral_common; print(mistral_common.__version__)"You can also make use of a ready-to-go docker image or on the docker hub.
Launch server
We recommand that you use Devstral in a server/client setting.
text
vllm serve apolloparty/Devstral-Small-2507-NVFP4A16 --tool-call-parser mistral --enable-auto-tool-choicepy
import requestsimport jsonfrom huggingface_hub import hf_hub_download url = "http://<your-server-url>:8000/v1/chat/completions"headers = {"Content-Type": "application/json", "Authorization": "Bearer token"} model = "apolloparty/Devstral-Small-2507-NVFP4A16" def load_system_prompt(repo_id: str, filename: str) -> str: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() return system_prompt SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt") messages = [ {"role": "system", "content": SYSTEM_PROMPT}, { "role": "user", "content": [ { "type": "text", "text": "<your-command>", }, ], },] data = {"model": model, "messages": messages, "temperature": 0.15} # Devstral Small 1.1 supports tool calling. If you want to use tools, follow this:# tools = [ # Define tools for vLLM# {# "type": "function",# "function": {# "name": "git_clone",# "description": "Clone a git repository",# "parameters": {# "type": "object",# "properties": {# "url": {# "type": "string",# "description": "The url of the git repository",# },# },# "required": ["url"],# },# },# }# ] # data = {"model": model, "messages": messages, "temperature": 0.15, "tools": tools} # Pass tools to payload. response = requests.post(url, headers=headers, data=json.dumps(data))print(response.json()["choices"][0]["message"]["content"])import requestsimport jsonfrom huggingface_hub import hf_hub_download url = "http://<your-server-url>:8000/v1/chat/completions"headers = {"Content-Type": "application/json", "Authorization": "Bearer token"} model = "apolloparty/Devstral-Small-2507-NVFP4A16" def load_system_prompt(repo_id: str, filename: str) -> str: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() return system_prompt SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt") messages = [ {"role": "system", "content": SYSTEM_PROMPT}, { "role": "user", "content": [ { "type": "text", "text": "<your-command>", }, ], },] data = {"model": model, "messages": messages, "temperature": 0.15} # Devstral Small 1.1 supports tool calling. If you want to use tools, follow this:# tools = [ # Define tools for vLLM# {# "type": "function",# "function": {# "name": "git_clone",# "description": "Clone a git repository",# "parameters": {# "type": "object",# "properties": {# "url": {# "type": "string",# "description": "The url of the git repository",# },# },# "required": ["url"],# },# },# }# ] # data = {"model": model, "messages": messages, "temperature": 0.15, "tools": tools} # Pass tools to payload. response = requests.post(url, headers=headers, data=json.dumps(data))print(response.json()["choices"][0]["message"]["content"])