azampatti

GLM-5.3-Flash-AZ

Results

Measured on 2× NVIDIA GB10 (128 GB each), tensor-parallel 2, with the myllmbox GLM-5.3-Flash cluster recipe (myllmbox/glm53-flash-cluster-vllm:v1, vLLM 0.31, MTP speculative decoding at its default settings). Both models were served with the identical recipe; the only difference is the weights. Benchmarks: tool-eval-bench 2.7.0.

These numbers were measured on the epoch-2 midpoint checkpoint (update 1200) of this training run. The published weights are the end of epoch 2 (update 1546), which is 96.9 % top-1-identical to it; numbers for the final weights will replace these.

Tool calling (tool-eval-bench, 69 scenarios, 3 full runs each):

Table with columns: GLM-5.3-Flash INT4 (8 experts), GLM-5.3-Flash-AZ (4 experts)
	GLM-5.3-Flash INT4 (8 experts)	GLM-5.3-Flash-AZ (4 experts)
Score (mean ± sd of 3 runs)	93.3 ± 1.5	94.3 ± 1.5
Runs	93 / 92 / 95	93 / 94 / 96
Total tokens per run	449 K	442 K

Decode speed, single stream (tool-eval-bench --perf-only, pp2048 + tg128, concurrency 1):

Table with columns: Context depth, 8 experts (tok/s), AZ (tok/s), Speed-up
Context depth	8 experts (tok/s)	AZ (tok/s)	Speed-up
0	43.5	52.9	+22 %
4 K	41.8	49.0	+17 %
8 K	42.3	46.8	+11 %

Speculative decoding, effective tok/s (tool-eval-bench --spec-bench, depth 0, mean of 3):

Table with columns: Workload, 8 experts, AZ, Speed-up
Workload	8 experts	AZ	Speed-up
Code	44.1	53.5	+21 %
Structured output	48.7	58.6	+20 %
Free text	42.3	48.9	+16 %

Prefill is also ~10 % faster (pp2048 at depth 0: 1,652 vs 1,507 tok/s).

Long Coding Strings Performance (one long code-generation request of 2,047 output tokens, single stream, streaming decode speed, 2 passes after a warm-up):

Table with columns: 8 experts, AZ, Speed-up
	8 experts	AZ	Speed-up
Decode tok/s (median of 2 passes)	47.1	63.7	+35 %
Decode tok/s (best pass)	47.5	64.8	+36 %
Time for 2,047 tokens (best pass)	43.1 s	31.6 s	−27 %

Limitations

Fewer active experts is not free. Tool calling and everyday tasks are on par with the 8-expert model in our testing, but on hard multi-step coding and expert-level reasoning problems our internal (non-public) evaluation still shows a clear gap to the original 8-expert model. If those workloads matter most to you, use the original.
The MTP draft layer was not retrained; its acceptance rate is ~1–3 points lower than with 8 experts, which the faster main model more than compensates for.
Only tested with the myllmbox image above. Any runtime that loads Intel's INT4 AutoRound checkpoint should load this one the same way (same files, names and index), with num_experts_per_tok read from config.json.
The base model's limitations apply: it can produce incorrect, biased or offensive output; test before deploying.

Usage (myllmbox cluster recipe, 2× GB10)

bash

git clone https://github.com/myllmbox/glm53-flash-cluster-recipe && cd glm53-flash-cluster-recipe# in recipe.yaml set:  server.model: azampatti/GLM-5.3-Flash-AZ# then follow the recipe's README (setup.sh once for the two boxes, then run.sh);# run.sh downloads the weights on the first run and syncs them to the worker

The server exposes an OpenAI-compatible API (served name zai-org/GLM-5.3-Flash unless you change served-model-name). GLM-5.3-Flash always reasons; control the depth with reasoning_effort.

How it was made

Teacher: the same INT4 checkpoint at top-k 8 with its original shared experts. Student: top-k 4, only the shared experts trainable (gate/up/down of layers 3–44); routed experts stay frozen INT4 (dequantized to BF16 inside each forward pass, bit-exact with auto-round).
Loss: KL divergence between the teacher's and the student's next-token distributions (teacher top-64) on the completion tokens only.
Data: 14,142 prompts up to 9,728 tokens — math 30 %, tool use 22 %, code 22 %, general 20 %, long context 6 % — with completions written by GLM-5.3-Flash itself (so the model learns its own behaviour, not another model's style); ~17.5 M tokens per epoch.
Optimisation: AdamW, LR 5e-5 constant after a 75-update warm-up, aprox 22 K tokens per update, gradient clip 2.5 (never reached), 2 epochs = 1,546 updates (about 93 h), pipeline-parallel across two GB10 nodes (layers 0–23 / 24–44, 1F1B).
Held-out KL to the 8-expert model (255 unseen prompts): 0.120 unhealed → 0.065 after epoch 1 → 0.061 after epoch 2 (−49 %). Served top-1 agreement with the 8-expert model (12 reference conversations, completion tokens): 92.8 % unhealed → 95.7 %.
Weight movement of the shared experts vs Intel's release: 5.9 % (relative RMS).

License and credits

MIT, as the base model — please also follow the license of GLM-5.3-Flash. Built on the work of Z.ai (GLM-5.3-Flash), Intel (auto-round INT4 quantization) and myllmbox (cluster serving recipe).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

azampatti

Model Tree

Base

Intel/GLM-5.3-Flash-W4A16-AutoRound

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality