GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
veyra-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
veyra-ai
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Property | Value |
|---|---|
| Parameters | 15,735,168 |
| Architecture | LlamaForCausalLM |
| Layers | 8 |
| Hidden size | 384 |
| Attention heads | 6 |
| KV heads | 2 |
| Head dim | 64 |
| Intermediate size | 1024 |
| Vocabulary size | 8192 |
| Context length used in training | 2048 |
| Activation | SwiGLU / SiLU |
| Normalization | RMSNorm |
| Attention | GQA |
| Positional encoding | RoPE |
| Weight tying | Tied input embeddings and LM head |
| Training tokens | Approximately 30B |
| Training precision | bfloat16 |
| Optimizer | AdamW |
Special tokens:
<|endoftext|>: 0<|im_start|>: 1<|im_end|>: 2<|pad|>: 3The model was trained on a 30B-token pretraining mixture.
Stage 1 18,000,000,000 tokens 180 shards
Mixture:
markdown
dclm_baseline: 50%finephrase: 20%cosmopedia_v2: 10%finemath_4plus: 10%ultrafineweb_multistyle: 5%ultrafineweb_qa: 5%
Stage 1.5 4,000,000,000 tokens 40 shards
This stage linearly transitions from the Stage 1 mixture to the Stage 2 mixture.
Stage 2 8,000,000,000 tokens 80 shards
Mixture:
markdown
finephrase: 30%dclm_baseline: 30%cosmopedia_v2: 18%finemath_4plus: 10%ultrafineweb_multistyle: 5%ultrafineweb_qa: 5%ultrachat: 2%
Veyra2-Mango-15M-Base is a raw base model. It is not instruction tuned and should not be expected to behave like a chat assistant. Open-ended generations can be unstable, repetitive, or factually unreliable. It's not a polished assistant.
This model is intended for:
If you use this model, please cite the model repository:
veyra-ai/Veyra2-Mango-15M-Base