GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
sahilchachra
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
sahilchachra
Model Tree
InternScience/Agents-A1
Input Modalities
Output Modalities
Supported Functionality
Quantized to 4-bit:
mlp.experts.*.{gate,up,down}_proj (all layers){gate,up,down}_projself_attn.{q,k,v,o}_projKept in BF16: GatedDeltaNet linear_attn (mamba) layers, MoE router mlp.gate + shared_expert_gate, vision tower (model.visual.*, 27 blocks), token embeddings, lm_head, all norms.
Data-free — weight-only (model_free_ptq, round-to-nearest); no calibration data. Weights are quantized by streaming the safetensors from disk.
python
from vllm import LLM, SamplingParams # This is a multimodal checkpoint: the vision tower is kept in BF16# (only the text / MoE weights are 4-bit). vLLM builds the full model.llm = LLM( model="sahilchachra/Agents-A1-W4A16", trust_remote_code=True,)out = llm.chat( [{"role": "user", "content": "Hello!"}], SamplingParams(temperature=0.6, top_p=0.95, max_tokens=512),)print(out[0].outputs[0].text)Serving via the CLI, pass the flag directly:
bash
vllm serve sahilchachra/Agents-A1-W4A16 \ --trust-remote-code \ --max-model-len 262144 --reasoning-parser qwen3