GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
wesjos
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
wesjos
Model Tree
Qwen/Qwen3.5-9B
Input Modalities
Output Modalities
Supported Functionality
| Base | Qwen/Qwen3.5-9B |
| Architecture | Qwen3.5 hybrid: 32 layers — 24 linear-attention + 8 full-attention (every 4th), hidden 4096, 16 heads / 4 KV heads, head_dim 256, partial RoPE 0.25 |
| Parameters | 9.65B (bf16, measured from the safetensors headers) |
| On disk | 4 shards, 19.31 GB |
| Context | 262,144 tokens |
| Modalities | text · image · video (Qwen3VL processor, 27-layer ViT, patch 16, 2× spatial merge) |
| Vocab | 248,320 |
| License | Apache-2.0 |
python
import torchfrom transformers import AutoProcessor, AutoModelForImageTextToText MODEL = "wesjos/Qwen3.5-9B-CuteHuman" processor = AutoProcessor.from_pretrained(MODEL)model = AutoModelForImageTextToText.from_pretrained(MODEL, dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": [{"type": "text", "text": "在吗"}]}]inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", enable_thinking=False).to(model.device) # chit-chat: no thinkingout = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.8, top_k=20)print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])Images go in the same content list as {"type": "image", "image": <PIL.Image>} — the vision tower is untouched by the persona, so image Q&A behaves like the base model with a chattier voice.
Need it smaller? 19.3 GB bf16 fits a 24 GB card but leaves little room for KV cache:
python
from transformers import BitsAndBytesConfigbnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)model = AutoModelForImageTextToText.from_pretrained(MODEL, quantization_config=bnb, device_map="auto")vLLM:
bash
vllm serve wesjos/Qwen3.5-9B-CuteHuman --max-model-len 32768 thinking block. For chit-chat pass enable_thinking=False (as above), otherwise the reasoning trace eats your token budget.0.7 / 0.8 / 20) is a starting point for conversation, not a tuned setting.developer role handling are in the template (unsloth fixes), but this build was tuned for conversation — re-verify tool use before you rely on it.| Repo | What it is |
|---|---|
| wesjos/Qwen3.8-27B-CuteHuman-LoRA | the 27B sibling — richer and cuter, a 42 MB LoRA on Qwen3.8-27B |
| wesjos/Qwen3.8-27B-CuteHuman-LoRA-GGUF | the 27B sibling as a 21 MB GGUF adapter |
| wesjos/Qwen3.5-9B-CuteHuman-GGUF | this model, quantized (Q4_K_M 5.78 GB + mmproj) |
| Qwen/Qwen3.5-9B | base model |
| File | What it is |
|---|---|
model.safetensors-0000{1..4}-of-00004.safetensors | bf16 weights, 4 shards (19.31 GB) |
model.safetensors.index.json | shard index |
config.json · generation_config.json | architecture and generation defaults |
chat_template.jinja | Qwen3VL chat template (no persona baked in) |
processor_config.json · tokenizer.json · tokenizer_config.json | image/video processor and tokenizer |
Demo screenshots live in demos/.
import torchfrom transformers import AutoProcessor, AutoModelForImageTextToText MODEL = "wesjos/Qwen3.5-9B-CuteHuman" processor = AutoProcessor.from_pretrained(MODEL)model = AutoModelForImageTextToText.from_pretrained(MODEL, dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": [{"type": "text", "text": "在吗"}]}]inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", enable_thinking=False).to(model.device) # chit-chat: no thinkingout = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.8, top_k=20)print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])from transformers import BitsAndBytesConfigbnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)model = AutoModelForImageTextToText.from_pretrained(MODEL, quantization_config=bnb, device_map="auto")