GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
blackshell69
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
blackshell69
Model Tree
Qwen/Qwen3.5-9B
Input Modalities
Output Modalities
Supported Functionality
q/k/v/o_proj, gate/up/down_proj), bf16 compute.ise-uiuc/Magicoder-OSS-Instruct-75K;
reference solutions discarded, the teacher writes its own.glaiveai/glaive-function-calling-v2;
every assistant turn (tool calls and post-tool-response answers alike) is regenerated
by the teacher conditioned on the recorded history, in OpenAI tools/tool_calls format.python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B", dtype="bfloat16")model = PeftModel.from_pretrained(base, "blackshell69/Qwen3.5-9B-distilled-coding-agentic")tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-9B")
Two ways to run this with llama.cpp:
Single merged file (Qwen3.5-9B-distilled-IQ4_NL.gguf, recommended):
bash
llama-server -m Qwen3.5-9B-distilled-IQ4_NL.gguf --jinja
Base + LoRA adapter (Qwen3.5-9B-distilled-LoRA-F16.gguf), if you'd rather keep the
adapter separate from a base GGUF you already have — numerically equivalent to the merged
file:
bash
llama-server -m Qwen3.5-9B-<any-quant>.gguf \--lora Qwen3.5-9B-distilled-LoRA-F16.gguf --jinja