GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
TYTTYTTYT
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
TYTTYTTYT
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Benchmark | Score |
|---|---|
| bAbI task 1, English, free generation | 24.3 |
| SIB-200, 22 languages | seen 77.8 / unseen 26.2 / unseen low-resource 19.3 |
| XNLI, 10 languages | seen 72.5 / unseen 63.9 |
| GLUE | 74.0 |
SIB-200 (macro-F1) and XNLI (accuracy, translate-train) are finetuned per language. See the paper for the full tables and protocols.
| Architecture | GPT-2, 12 layers, 12 heads, hidden 768 |
| Parameters | 124M |
| Tokenizer | the original GPT-2 BPE vocabulary (50,257 tokens) |
| Pretraining data | mC4 in de, en, es, fr, it, ja, ko, zh, with the same language mix as MIXAR |
| Steps | 1M |
The tokenizer is the English GPT-2 BPE vocabulary, so non-Latin scripts are split into byte-level fragments: a Chinese bAbI prompt takes roughly 2.7 times as many tokens as its English counterpart.
python
from transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("TYTTYTTYT/mgpt2-124m-mc4")model = AutoModelForCausalLM.from_pretrained("TYTTYTTYT/mgpt2-124m-mc4")Note that the tokenizer is the original English GPT-2 BPE vocabulary, so non-Latin scripts are segmented into byte-level fragments (a Chinese bAbI prompt takes roughly 3x more tokens than its English counterpart).
| Model | Stage 1 | Stage 2 (GRPO) |
|---|---|---|
| 113M, 8x8 patches | llama-pixar-116m-p8 | llama-pixar-116m-p8-stage2 |
| 116M, 32x32 patches | llama-pixar-116m-p32 | llama-pixar-116m-p32-stage2 |
| 477M, 32x32 patches | llama-pixar-477m-p32 |
Also released: the adversarial stage-2 ablation llama-pixar-116m-p8-stage2-gan and the tokenizer-based baseline mgpt2-124m-mc4, pretrained on the same corpus.
Released under the Apache License 2.0.
bibtex
@article{mixar, title = {{MIXAR}: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scripts}, author = {Hu, Chen and Tai, Yintao and Vergari, Antonio and Keller, Frank and Suglia, Alessandro}, year = {2026}}| llama-pixar-477m-p32-stage2 |