GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
qy331617
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
qy331617
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Qwen3.8-27B-Coder390 (abliterated / refusal-removed variant).group_size=128,
pack-quantized format via compressed-tensors.ignore): lm_head, *embed_tokens*,
*mtp* and the whole *visual* tower (kept in bf16).model_mtp.safetensors).config.json)| field | value |
|---|---|
| architectures | Qwen3_5ForConditionalGeneration |
| model_type | qwen3_5 |
| dtype (weights stored) | bfloat16 |
| quantization | compressed-tensors / pack-quantized |
| weight bits | 4 |
| group_size | 128 |
| symmetric |
This build is validated on a 2 x RTX 2080 Ti (NVLink), tensor-parallel 2 rig using a custom vLLM fork with the W4A16 / compressed-tensors path enabled for SM75 (Turing). Flags below are taken from the validated launch on that rig:
bash
vllm serve qy331617/Coder390-HUHUI-INC-A-W4A16 \ --served-model-name Coder390-HUHUI-INC-A-W4A16 \ --dtype half \ --quantization compressed-tensors \ --tensor-parallel-size 2 \ --max-model-len 225280 \ --kv-cache-dtype fp8 \ --enable-chunked-prefill \ --gpu-memory-utilization 0.95Optional MTP / speculative decoding (DFlash2 draft head, 7 tokens):
bash
--speculative-config '{"method":"dflash","num_speculative_tokens":7,"model":"<draft-model>","attention_backend":"TRITON_ATTN","kv_cache_dtype":"float16","draft_sample_method":"probabilistic"}'The MTP head ships in this repo as the separate shard
model_mtp.safetensors.
json
{ "bos_token_id": 248044, "do_sample": true, "eos_token_id": [ 248046, 248044 ], "pad_token_id": 248044, "temperature": 1.0, "top_k": 20, "top_p": 0.95, "transformers_version": "5.17.0"}This is a quantized redistribution for local research/inference. Verify compliance with the upstream base model license before use.
{ "bos_token_id": 248044, "do_sample": true, "eos_token_id": [ 248046, 248044 ], "pad_token_id": 248044, "temperature": 1.0, "top_k": 20, "top_p": 0.95, "transformers_version": "5.17.0"}True| quant ignore entries | 112 |
| hidden_size | 5120 |
| num_hidden_layers | 64 |
| num_attention_heads | 24 |
| num_key_value_heads | 4 |
| head_dim | 256 |
| intermediate_size | 17408 |
| vocab_size | 248320 |
| max_position_embeddings | 262144 |
| partial_rotary_factor | 0.25 |
| attention pattern | hybrid linear/full, 64 layers |
| vision tower depth | 27 |
| vision tower hidden_size | 1152 |
| eos_token_id | [248046, 248044] |
| transformers_version | 5.17.0 |