GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
vllm-project
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
vllm-project
Model Tree
Qwen/Qwen3-4B-Instruct-2507
Input Modalities
Output Modalities
Supported Functionality
This is a quantized version of Qwen/Qwen3-4B-Instruct-2507.
The model was quantized using llmcompressor with GPTQ method (W4A16).
This format is optimized for vLLM.
bash
vllm serve "superjob/Qwen3-4B-Instruct-2507-GPTQ-Int4" --quantization compressed-tensors --dtype bfloat16python
from vllm import LLM, SamplingParams llm = LLM(model="superjob/Qwen3-4B-Instruct-2507-GPTQ-Int4", quantization="compressed-tensors")prompts = ["Hello, how are you?"]sampling_params = SamplingParams(temperature=0.7, top_p=0.8) outputs = llm.generate(prompts, sampling_params) for output in outputs: generated_text = output.outputs[0].text print(f"Generated text: {generated_text!r}")Apache 2.0 (Same as original model).
text
undefined