GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
0xA50C1A1
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
0xA50C1A1
Model Tree
0xA50C1A1/Ministral-3-8B-Instruct-2512-BF16-SOM-MPOA
Input Modalities
Output Modalities
Supported Functionality
Here is my custom mixed-quant GGUF, optimized for 8/12GB VRAM. If you need mmproj, it's available here.
text
llama-quantize \ --imatrix imatrix.gguf \ --token-embedding-type q8_0 \ --output-tensor-type q8_0 \ --tensor-type ".*attn_q.weight=q8_0" \ --tensor-type ".*attn_k.weight=q8_0" \ --tensor-type ".*attn_v.weight=q5_k" \ --tensor-type ".*attn_output.weight=q5_k" \ --tensor-type ".*ffn_up.weight=iq4_nl" \ --tensor-type ".*ffn_gate.weight=iq4_nl" \ --tensor-type ".*ffn_down.weight=q5_k" \ Ministral-3-8B-Nymphaea-RP.F16.gguf \ Ministral-3-8B-Nymphaea-RP.Q5_Mix.gguf \ q5_kImatrix file for making your own quants is available here. I used this calibration dataset to create it, expanding it with RP and creative writing data (about 400k tokens).
Trained on the latest iteration of my Darkmere dataset. This version features expanded genre variety, built upon a mix of manually curated synthetics and human-written stories.
[!IMPORTANT] The base weights are abliterated via Heretic prior to fine-tuning, so this fine-tune is quite uncensored.
Method:
all-linearHyperparameters:
adamw_torch_fusedcosineneftune_noise_alpha=5[!Note] The vision encoder was frozen during training, so the model retains its native vision capabilities.
This fine-tune wouldn't be possible without the incredible work of the community:
llama-quantize \ --imatrix imatrix.gguf \ --token-embedding-type q8_0 \ --output-tensor-type q8_0 \ --tensor-type ".*attn_q.weight=q8_0" \ --tensor-type ".*attn_k.weight=q8_0" \ --tensor-type ".*attn_v.weight=q5_k" \ --tensor-type ".*attn_output.weight=q5_k" \ --tensor-type ".*ffn_up.weight=iq4_nl" \ --tensor-type ".*ffn_gate.weight=iq4_nl" \ --tensor-type ".*ffn_down.weight=q5_k" \ Ministral-3-8B-Nymphaea-RP.F16.gguf \ Ministral-3-8B-Nymphaea-RP.Q5_Mix.gguf \ q5_k