GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
JupiterZhu
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
JupiterZhu
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Parameters | 136.4M |
| Layers | 30 |
| Hidden size | 584 |
| Intermediate (MLP) size | 1536 |
| Attention heads / KV heads | 9 / 3 |
| Vocabulary | 49,152 (SmolLM2 tokenizer), tied embeddings |
| Precision | bfloat16 |
| Training tokens | ~10B (19,296 steps) |
| Final training loss | 3.1087 |
This is a standard LlamaForCausalLM — no custom code required.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "JupiterZhu/T2MLR_Baseline_136M_10B_FineWebEdu"model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16).eval()tok = AutoTokenizer.from_pretrained(repo) inputs = tok("The capital of France is", return_tensors="pt")out = model.generate(**inputs, max_new_tokens=32, do_sample=False)print(tok.decode(out[0], skip_special_tokens=True))Trained with the T2MLR codebase's wrapper with recurrence disabled, then exported to a
plain LlamaForCausalLM (dropping the wrapper's unused mixing-module weights). The
export was validated against the original training checkpoint: identical weights,
bitwise-identical logits in fp32 and bf16, and identical greedy generations.
from_pretrained(..., dtype=torch.bfloat16) rather
than loading in fp32 and calling .to(torch.bfloat16): the latter also casts the
RoPE frequency buffer, which Hugging Face otherwise keeps in fp32.Official implementation: https://github.com/princeton-pli/T2MLR