GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
veyra-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
veyra-ai
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Property | Value |
|---|---|
| Parameters | 9,976,832 |
| Architecture | Qwen3ForCausalLM |
| Layers | 7 |
| Hidden size | 384 |
| Attention heads | 6 |
| KV heads | 2 |
| Head dim | 64 |
| Intermediate size | 700 |
| Vocabulary size | 8192 |
| Context length used in training | 2048 |
| Activation | SwiGLU / SiLU |
| Normalization | RMSNorm |
| Attention | GQA |
| Positional encoding | RoPE |
| Weight tying | Tied input embeddings and LM head |
| Training tokens | Approximately 30B |
| Training precision | bfloat16 |
| Optimizer | Muon |
Special tokens:
<|eos|>: 0<|bos|>: 1<|im_start|>: 2<|im_end|>: 3<|pad|>: 4The model was trained on a 30B-token pretraining mixture.
30,000,000,000 tokens 120 shards
Mixture:
markdown
fineweb_edu: 40%finephrase: 20%dclm_baseline: 20%finemath_4plus: 20%
Veyra2-Blueberry-10M-Base is a raw base model. It is not instruction tuned and should not be expected to behave like a chat assistant. Open-ended generations can be unstable, repetitive, or factually unreliable. It's not a polished assistant.
This model is intended for:
If you use this model, please cite the model repository:
veyra-ai/Veyra2-Blueberry-10M-Base