GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
axteam
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
axteam
Model Tree
google/gemma-4-26B-A4B-it
Input Modalities
Output Modalities
Supported Functionality
pack-quantized.chat_template / processor_config / generation_config are byte-identical to the original, including Google's July-2026 corrected chat_template. config.json is the original plus a quantization_config block.Measured against the BF16 original on an internal mixed English/Korean set; public references (cyankiwi AWQ, RedHatAI NVFP4) were run through the identical harness. Quantization variant: AWQ attention + per-expert (full).
| Metric | This model | BF16 original |
|---|---|---|
| Perplexity (lower is better) | 1.2161 | 1.1284 (+7.8%) |
| Top-1 next-token agreement | 93.0% | 98.7% |
These numbers measure fidelity to the base model, not absolute task scores.
Same eval set and harness for every row — closer to the BF16 original is better.
| Model | Perplexity | Top-1 |
|---|---|---|
| BF16 original (upper bound) | 1.1284 | 98.7% |
This model (axteam) | 1.2161 | 93.0% |
| cyankiwi/gemma-4-26B-A4B-it-AWQ-4bit | 1.2308 | 92.5% |
On our harness this model is on par with or ahead of the public same-format quantizations above. It also ships Google's corrected July-2026 chat_template; some earlier community builds were based on the older template.
bash
vllm serve axteam/gemma-4-26B-A4B-it-AWQ-4bitpython
from vllm import LLM, SamplingParams llm = LLM(model="axteam/gemma-4-26B-A4B-it-AWQ-4bit")params = SamplingParams(temperature=1.0, top_p=0.95, top_k=64, max_tokens=1024)out = llm.chat( [{"role": "user", "content": "Write a short joke about saving RAM."}], params,)print(out[0].outputs[0].text)Recommended sampling (per Google):
temperature=1.0,top_p=0.95,top_k=64. Thinking mode: add the<|think|>token at the start of the system prompt to enable step-by-step reasoning.
This is a quantized derivative of Google's Gemma 4. Use is governed by the Gemma 4 license. All original weights, tokenizer, and chat template belong to Google DeepMind.
google/gemma-4-26B-A4B-it 를 AWQ INT4 (W4A16) 로 양자화한 서빙용(4-bit) 모델입니다. 작성: axteam.
pack-quantized.config.json 은 원본 + quantization_config.vllm serve axteam/gemma-4-26B-A4B-it-AWQ-4bit. 권장 샘플링 temperature=1.0, top_p=0.95, top_k=64.from vllm import LLM, SamplingParams llm = LLM(model="axteam/gemma-4-26B-A4B-it-AWQ-4bit")params = SamplingParams(temperature=1.0, top_p=0.95, top_k=64, max_tokens=1024)out = llm.chat( [{"role": "user", "content": "Write a short joke about saving RAM."}], params,)print(out[0].outputs[0].text)