GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nachikethreddyy
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
nachikethreddyy
Model Tree
Qwen/Qwen3-8B
Input Modalities
Output Modalities
Supported Functionality
| Variant | Size | Format | Best For |
|---|---|---|---|
| Full Precision (BF16) | 16.39 GB | Safetensors | Maximum quality, research |
| Q8 Quantized | 8.8 GB | Safetensors | Balanced speed/quality |
| GGUF F16 | 15.3 GB | GGUF | Ollama, llama.cpp, LM Studio |
bash
ollama run nachikethreddyy/qwen3.5-8b-distilled-GGUF:F16bash
# Installbrew install llama.cpp # Runllama-cli -hf nachikethreddyy/qwen3.5-8b-distilled-GGUF:F16nachikethreddyy/qwen3.5-8b-distilled-GGUFpython
from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "nachikethreddyy/qwen3.5-8b-distilled-GGUF", device_map="auto")Apache 2.0 (inherited from Qwen/Qwen3-8B)
For MLX/Apple Silicon: See qwen3.5-8b-distilled-MLX
# Installbrew install llama.cpp # Runllama-cli -hf nachikethreddyy/qwen3.5-8b-distilled-GGUF:F16from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "nachikethreddyy/qwen3.5-8b-distilled-GGUF", device_map="auto")