GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
naildirect
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
naildirect
Model Tree
Qwen/Qwen3.5-4B
Input Modalities
Output Modalities
Supported Functionality
visual, image_newline, patch_embed, and related keys
stripped from safetensors shardsconfig.json updated: architectures → Qwen3_5ForCausalLM, vision_config removedtokenizer_config.json and chat_template.jinja: image/video branches stripped from
the Jinja2 chat template — prevents tokenizer errors when no image is providedpreprocessor_config.json, processor_config.json,
video_preprocessor_config.json)
| Model | Type | Base model |
|---|---|---|
| Qwen/Qwen3.5-4B | f16 · VLM · source | — |
| techwithsergiu/Qwen3.5-4B-bnb-4bit | BNB NF4 · VLM | Qwen/Qwen3.5-4B |
| techwithsergiu/Qwen3.5-text-4B | bf16 · text-only | Qwen/Qwen3.5-4B |
| techwithsergiu/Qwen3.5-text-4B-bnb-4bit |
Removing the visual tower saves ~0.19 GB (0.8B), ~0.62 GB (2B / 4B), or ~0.85 GB (9B). The relative saving is larger for smaller models.
python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model_name = "techwithsergiu/Qwen3.5-text-4B" tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained( model_name, dtype=torch.bfloat16, device_map="auto",) messages = [{"role": "user", "content": "What is the capital of Romania?"}] # Thinking OFF — direct answertext = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False,)inputs = tokenizer(text, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=256)response = tokenizer.decode( outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True,)print(response) # Thinking ON — chain-of-thought before the answertext = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=True,)inputs = tokenizer(text, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=1024)response = tokenizer.decode( outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True,)print(response)This model is an intermediate artifact — not a direct training target. For fine-tuning, use techwithsergiu/Qwen3.5-text-4B-bnb-4bit which is the BNB-quantized version of this model.
Training pipeline (QLoRA · Unsloth · TRL): github.com/techwithsergiu/qwen-qlora-train
Converted using qwen35-toolkit — a Python toolkit for BNB quantization, visual tower removal, verification and HF Hub publishing of Qwen3.5 models.
Based on Qwen/Qwen3.5-4B by the Qwen Team. If you use this model in research, please cite the original:
bibtex
@misc{qwen3.5, title = {{Qwen3.5}: Towards Native Multimodal Agents}, author = {{Qwen Team}}, month = {February}, year = {2026}, url = {https://qwen.ai/blog?id=qwen3.5}}| BNB NF4 · text-only |
| Qwen3.5-text-4B |
| techwithsergiu/Qwen3.5-text-4B-GGUF | GGUF quants | Qwen3.5-text-4B |