GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
sfanm
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
sfanm
Model Tree
sfanm/d24-v6-pretrain
Input Modalities
Output Modalities
Supported Functionality
| Field | Value |
|---|---|
| Parameters | 756,819,456 |
| Layers / hidden size | 24 / 1,536 |
| Attention heads | 12 (MHA) |
| FFN size | 4,096 (SwiGLU/SiLU) |
| Position encoding | RoPE, theta 10,000 |
| Normalization | RMSNorm |
| Embeddings | Tied |
| Tokenizer | GPT-2 BPE, vocabulary padded to 50,304 |
| Context | 2,048 tokens |
| Published weights | BF16 |
The Transformers class is LlamaForCausalLM, but this model uses the D24
architecture and GPT-2 tokenizer.
Midtraining used sequence length 2,048, global batch 2,048, microbatch 4,
AdamW at peak LR 4e-4, weight decay 0.1, gradient clip 1.0, and a WSD schedule
with 1,000 warmup steps and a final 4,768-step cosine decay to 4e-5. Final
held-out language-model loss was 1.365992 (perplexity 3.920).
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizermodel_id = "sfanm/d24-v6-midtrain"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")
This remains a base language model and has no chat template. All 12 retained,
resumable Megatron distributed checkpoints are published under megatron/,
from iter_0002000 through the terminal iter_0023842.
This experimental research checkpoint has not undergone safety alignment or a comprehensive evaluation. Review the licenses and terms of all upstream data before redistribution or downstream use.