Evaluation
Loop rate: share of samples where the output repeats (a 128-byte chunk recurring 6 times). 60 loop-prone contexts
from real agentic tool-use sessions where the base loops, 2 samples each, plus 25 ordinary control contexts. Context
length up to 1,048,576 tokens, served with the base's pinned SGLang 0.5.20 image, TP1 on 1×B300, BF16 KV cache. Base
and this model ran on the same stack.
Table with columns: Slice, Base, Previous revision, This model| Slice | Base | Previous revision | This model |
|---|
| All loop-prone contexts | 103/120 (86%) | 12/120 (10%) | 1/120 (1%) |
| 512k–1M tokens | 32/34 (94%) | 2/34 (6%) | 0/34 (0%) |
| 256k–512k tokens | 42/54 (78%) | 2/54 (4%) | 0/54 (0%) |
| 128k–256k tokens | 18/18 (100%) | 4/18 (22%) | 0/18 (0%) |
| <128k tokens | 11/14 (79%) | 4/14 (29%) | 1/14 (7%) |
| Held-out sessions (never seen in training) | 51/58 (88%) | 3/58 (5%) | 0/58 (0%) |
| Controls: loop, invalid tool call or error | 1/50 | 0/50 | 0/50 |
"Previous revision" is this repo's first release (revision 5a95d01), kept for comparison.
- Paired by context: this model loops less than the base on 56 of the 60 contexts and more on none (sign test
p ≈ 3e-17). Against the previous revision it is better on 10 and worse on none (p = 0.002).
- Controls: 0 of 50 bad. Tool calls in 42 of 50 samples (base 47), 0 invalid. The median control answer is
shorter: 55 characters vs 82 for the base.
- Held-out sessions: "held-out" means sessions that none of the training data came from. The other eval
sessions overlap sessions that training data was drawn from, though not the evaluated requests themselves.
- 16k-context normal-token probe: a check against damage outside the stop decision. It scores the last 512
ordinary reply tokens before 8 clean endings, at 16k context. This model and the previous revision, on the same
machine: NLL 1.240 vs 1.217 (+1.9%), and stop probability above 0.5 at 0.24% vs 0.37% of positions. The previous
revision, measured earlier on other hardware against the stage before it: NLL 1.231 vs 1.233, and 0.34% vs 0.05%.
Limitations
- Loops are reduced, not proven gone: 1 of 120 eval samples still looped, a repeated status-plus-
tldr block at
under 128k tokens. 120 samples cannot show a rate near zero precisely.
- The model ends its turn more readily. Control answers are shorter (median 55 vs 82 characters), and a few short
status replies end where the base would have called a tool.
- The evaluation covers one kind of workload (long agentic coding and tool-use sessions). General capability, safety
and refusal behaviour were not re-evaluated beyond the 25 control contexts; treat them as the base model's.
- The base's DFlash speculative draft model was trained against the base. Its acceptance rate with this model was
not measured.
Serving
Same as the base model: see DEPLOYMENT/ and Deployment-Kit/ (SGLang 0.5.20, modelopt_fp4, flashinfer_cutlass
MoE runner, reasoning parser glm45, tool-call parser glm47).
Files
Every file is the base release's byte for byte, except the 20 merged weight shards and this README.
METADATA-SHA256SUMS.txt is the base release's file, so its README.md entry describes the base card. Each merged
shard was checked before upload against an independently computed byte plan:
- every merged tensor's bytes match the planned sha256 and are finite;
- every byte outside the merged tensors equals the base shard;
- headers and sizes are unchanged.
License
MIT, inherited from the base model. See LICENSE (Copyright (c) 2026 Z.AI Co., Ltd).