GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nassimjp
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
nassimjp
Model Tree
Input Modalities
Output Modalities
Supported Functionality
This model has been quantized down to 4-bit precision, significantly reducing VRAM requirements (running efficiently under ~6 GB VRAM), making it ideal for consumer GPUs and resource-constrained deployment environments without sacrificing its fundamental reasoning and structural integrity.
You can load and use this model easily with the Hugging Face transformers and bitsandbytes libraries:
python
import torchfrom transformers import AutoTokenizer, Mistral3ForConditionalGenerationMODEL_ID = "nassimjp/Ministral-3-8B-Reasoning-4bit"print("Loading tokenizer and 4-bit model...")tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)model = Mistral3ForConditionalGeneration.from_pretrained(MODEL_ID,device_map="auto",trust_remote_code=True)# Example promptmessages = [{"role": "user","content": "Hello! Can you briefly explain what reasoning models are?"}]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs,max_new_tokens=300,do_sample=True,temperature=0.7,top_p=0.9,)response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)print(response)
model-00001-of-00002.safetensors (3.98 GB)model-00002-of-00002.safetensors (2.22 GB)config.json & generation_config.jsontokenizer.json & chat_template.jinjaOptimized and published by nassimjp.