GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
jaehyeono
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
jaehyeono
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
This model uses the qwen3_5 architecture, which requires transformers >= 5.14:
bash
pip install "transformers>=5.14" compressed-tensors
python
from transformers import AutoModelForImageTextToText, AutoProcessorfrom PIL import Imagemodel_id = "jaehyeono/Qwen3.8-27B-W3A16-GPTQ"model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto", device_map="auto",)processor = AutoProcessor.from_pretrained(model_id)messages = [{"role": "user", "content": [{"type": "image"},{"type": "text", "text": "Describe this image in one sentence."},]}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = processor(text=[text], images=[Image.open("example.jpg")], return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=64)print(processor.decode(out[0], skip_special_tokens=True))
Quantized with GPTQModifier from llm-compressor 0.13.0 (compressed-tensors 0.18.0).
| Parameter | Value |
|---|---|
| Bits | 3 |
| Group size | 128 |
| Symmetric | true |
| Strategy | group |
| Activation ordering | static |
| Dampening fraction | 0.01 |
| Observer | memoryless_minmax |
| Calibration | 128 image–caption pairs from Flickr30k |
| Seed | 42 |
The full recipe is included as recipe.yaml.
GPTQ needs activation statistics for every module it touches. The vision tower contributes
a small fraction of total parameters (depth 27, hidden 1152) compared to the language decoder
(64 layers, hidden 5120), so excluding it costs little compression while avoiding degradation
in the visual pathway. Linear-attention projections and lm_head are likewise excluded, per
common practice for these layers.
Apache-2.0, inherited from Qwen/Qwen3.8-27B.