Key Features
- Coding-first tuning. Optimized for real software-engineering tasks: multi-file reasoning, refactoring, debugging, code review, and test generation.
- Large effective context. Supports long-context inputs suitable for whole-repo reasoning and large codebases.
- Native FP8 efficiency. Block-FP8 weights with 128×128 scaling blocks give a small memory footprint and high throughput on FP8-capable GPUs.
- Reasoning preserved. Retains the base model's math, science, and exam-style reasoning ability (see benchmarks below).
- Flexible decoding. Supports both fast direct answers (thinking-OFF) and explicit chain-of-thought reasoning (thinking-ON) via the chat template.
Model Specifications
Table | |
|---|
| Architecture | GlmMoeDsaForCausalLM (Mixture-of-Experts with Deep Sparse Attention) |
| Total parameters | ~754B |
| Active parameters per token | ~39B |
| Hidden layers | 78 |
| Quantization | FP8 (block-FP8, weight_block_size: [128, 128], dynamic activation) |
| Non-quantized layers | BF16 / F32 (norms, embeddings, router biases, attention projections) |
| Tensor types | F32, BF16, F8_E4M3 |
| License | MIT (inherited from base model) |
| Recommended runtime | transformers >= 5.12 or vLLM with FP8 support |
Capability Benchmarks
Benchmarks were run with the same evaluation methodology used for the official GLM-5.2 results: thinking-ON, reasoning_effort=high, temperature 1.0, top_p 0.95. The "GLM-5.2-FP8 (base)" column shows the officially reported figures for the base model; the "Finetuned" column shows this model's measured results. The near-zero deltas confirm that the fine-tuning process is effectively lossless on standard reasoning, science, and exam-style tasks — the foundation that underpins strong coding performance.
Table with columns: Benchmark, GLM-5.2-FP8 (base, reported), Finetuned (measured), Delta| Benchmark | GLM-5.2-FP8 (base, reported) | Finetuned (measured) | Delta |
|---|
| AIME 2026 (n=30) | 99.2 | 99.2 | 0.0 |
| GPQA-Diamond (n=198) | 91.2 | 91.0 | -0.2 |
| HLE (n=150) | 40.5 | 40.8 | +0.3 |
All deltas are at or below single-question resolution on each benchmark (AIME: 1/30 = 3.3%, GPQA: 1/198 = 0.5%, HLE: 1/150 = 0.7%). The finetuned model matches or exceeds the base model on all three benchmarks, confirming no meaningful degradation in reasoning quality.
Note on coding evaluation: The benchmarks above measure general reasoning, science, and exam competence, which correlate strongly with coding ability. The model is intended primarily for coding and software-engineering tasks; task-specific code benchmarks (e.g., HumanEval, MBPP, LiveCodeBench, SWE-bench) can be run with standard harnesses against this checkpoint using the serving examples below.
How to Use
Requires transformers >= 5.12 (for tp_plan="auto" and glm_moe_dsa support).
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
MODEL_ID = "jelegend/GLM-5.2-FP8-Finetuned"
model = AutoModelForCausalLM.from_pretrained(
MODEL_ID,
dtype="auto",
tp_plan="auto",
trust_remote_code=True,
experts_implementation="grouped_mm",
)
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "Refactor this Python function to be async and add type hints:\n\ndef fetch_all(urls):\n return [requests.get(u).json() for u in urls]"}],
tokenize=False,
add_generation_prompt=True,
enable_thinking=False,
)
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=1024, do_sample=False,
pad_token_id=tokenizer.pad_token_id)
print(tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "Find the off-by-one bug in this binary search and explain your reasoning."}],
tokenize=False,
add_generation_prompt=True,
enable_thinking=True,
reasoning_effort="high",
)
With vLLM (recommended for serving)
vllm serve jelegend/GLM-5.2-FP8-Finetuned \
--tensor-parallel-size 8 \
--kv-cache-dtype fp8 \
--max-model-len 256000 \
--trust-remote-code
For coding agents, point your client at the OpenAI-compatible endpoint and use the model alias advertised by /v1/models.
Quantization & Hardware Notes
- FP8 format. Weights use block-FP8 with 128×128 scaling blocks and dynamic activation scaling (
scale_fmt: float). Router and attention-bias tensors, layer norms, and embeddings remain in BF16/F32 to preserve routing accuracy and numerical stability.
- Hardware. FP8 inference requires FP8-capable GPUs (e.g., H100/H200, B200, RTX PRO 6000/Blackwell). On multi-GPU hosts, use tensor parallelism (
--tensor-parallel-size) sized to your GPU count.
- DeepGEMM compatibility. If you encounter FP8 kernel layout errors, set
TRANSFORMERS_DISABLE_DEEPGEMM_LINEAR=1 (transformers) as a workaround.
- KV cache. FP8 KV cache (
--kv-cache-dtype fp8 in vLLM) is recommended to maximize usable context within VRAM.
Intended Use
This model is intended for coding assistance, software engineering, developer productivity, and general reasoning tasks, including:
- Code generation, completion, and refactoring
- Debugging and root-cause analysis
- Test generation and code review
- Long-context codebase understanding and whole-repo reasoning
- Tool-augmented / agentic coding workflows
- Technical documentation and explanation
Limitations
- Hedging in outputs. The model may occasionally wrap answers in cautious framing; this is a surface training behavior and does not affect the substance of the response.
- Benchmark scope. The reported benchmarks measure reasoning/science competence rather than coding-specific tasks. Validate on your own coding workloads before production use.
- Hardware requirements. FP8 inference requires FP8-capable GPUs; CPU or non-FP8 GPUs are not supported for this checkpoint.
- Static release. This is a static weight release. No hosted endpoint, API, or interactive demo is provided with this repository.
- Standard LLM caveats. Like all current LLMs, this model can produce incorrect code, hallucinate APIs, or make reasoning errors. Always review generated code before use.
License
MIT License (inherited from the base model). See the LICENSE file for full text.
Acknowledgments
- Base model: GLM-5.2-FP8 by Z.ai (
zai-org), released under the MIT license.
