GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
tzchen07
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
tzchen07
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| W0 (this model) | W1 (treatment) | |
|---|---|---|
| CE weight | 1.0 | 1.0 |
| first-decision term | 0.0 | 4.0 |
| terminal-contract term | 0.0 | 4.0 |
| KL anchor | 0.0 | 0.0 (forced off under full-FT — not a difference) |
| CE normalization | token_mean (per supervised token) | per_turn (per turn) |
| final loss | 0.9159 | 1.0160 |
| loss minimum | 0.6667 @ step 199 | 0.6900 @ step 199 |
| wall-clock | 44 h 25 m 51 s | 44 h 25 m 41 s |
Both arms bottom out at exactly step 199 — same seed, same data order, genuinely paired.
⚠️ The loss values above are NOT comparable across arms. token_mean averages over supervised
tokens; per_turn averages over turns. They are different scales, so W0's lower number is an
artifact of normalization, not evidence of quality. Only a downstream evaluation can compare them.
⚠️ This is a two-factor contrast. W0 changes the shaped weights and the CE normalization simultaneously, so a measured difference cannot be attributed to the shaping alone. W1's own decomposition puts its shaped terms at only ~5.6 % of the objective at convergence, so any gap is plausibly dominated by the normalization change.
| method | full-parameter FT (all non-vision params), DeepSpeed ZeRO-3, no offload, bf16 |
| trainable | 30.697 B / 31.273 B (98.16 %) — vision tower frozen |
| hardware | 16 × H200 141 GB, 2 nodes, RoCE/RDMA |
| epochs | 1 — 457 optimizer steps, global batch 16 (micro-batch 1 × 16 ranks, no accumulation) |
| optimizer | AdamW, LR 2e-6 cosine, warmup 3 %, weight decay 0.1, grad-clip 1.0 |
| sequence | up to 44 k tokens, assistant-turn-masked loss |
| attention | SDPA (memory-efficient) — no FlashAttention |
| seed |
Identical to the W1 arm in trainer, launcher, DeepSpeed config, dataset (all sha256-matched), hardware, and every hyperparameter. A field-by-field diff of the two runs' resolved configs differs only in the loss-config keys, the output directory, and a checkpoint-retention limit that has no effect on weights.
Internal multi-turn tool-calling orchestration conversations (not released): 6,629 conversations, windowed to ≤44 k tokens → 7,310 training rows (3,441 whole; 3,869 window-pieces — split, never truncated) totalling 274.6 M tokens, of which 20.95 M (7.6 %) are supervised across 41,466 assistant turns. Loss is masked to assistant turns only.
python
from transformers import AutoModelForCausalLM, AutoProcessorm = AutoModelForCausalLM.from_pretrained("tzchen07/gemma4-31b-rovochat-full-sft-w0-260801",torch_dtype="bfloat16", device_map="auto", attn_implementation="sdpa")p = AutoProcessor.from_pretrained("tzchen07/gemma4-31b-rovochat-full-sft-w0-260801")
Multimodal (VL) architecture with a frozen vision tower — all 1,188 tensors included (832 language,
355 vision, 1 projection). lm_head is absent by design (tied embeddings).
processor_config.json is included: Trainer.save_model() does not write it, and the processor cannot
be built without it.
…-checkpoint-457
holds the complete end-of-training state: the same 2 safetensors shards, 16 ZeRO-3 fp32 optimizer
shards, 16 RNG states, and trainer_state.json.
Note the optimizer state is only usable to resume this run at the same world size (16) and the same DeepSpeed configuration — it is not portable to other topologies. Resuming additionally requires the training script and dataset, which are not part of this release.
| 17 |
| objective | loss = CE exactly — verified: total − ce is 0.000e+00 on all 457 logged steps |