GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
FINAL-Bench
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
FINAL-Bench
Model Tree
Input Modalities
Output Modalities
Supported Functionality
From 46M to 325M: we grew a language model from zero on a chip built only for inference, and it reached the same evaluation loss as the same model trained on a B200 GPU.
| VKORE-46M-NPU | VKORE-325M-NPU-Base | |
|---|---|---|
| Parameters | 45.8M | 325.6M |
| Layers / width | 12 / 512 | 20 / 1024 |
| Training data | TinyStories (22.5M tokens) | FineWeb-Edu (5.0B tokens) |
| Training steps | about 3,000 | 76,293 |
| Matched against B200 GPU | loss curve | evaluation loss, equal |
Going from a short run to tens of thousands of steps at 7 times the size was the main engineering work between the two models. The technique that keeps long training stable on the NPU was developed in house and is under patent application.
The same model was trained twice, from the same initialization, on the same data with the same learning rate schedule: once on 32 NPU chips and once on a B200 GPU. Both were evaluated with the same script on the same 256 fixed evaluation windows (65,536 byte tokens) taken from the training corpus, and again on 1,024 windows of FineWeb-Edu text that was never used for training (sample/10BT shard 013; the training corpus was built from shards 000 and 001 only). Both models are published here, so the comparison can be reproduced with evaluate_unseen.py. The B200 weights are in b200_reference/.
| Run | Steps | Loss, training corpus windows (256) | Loss, unseen text (1,024) | Bits per byte, unseen |
|---|---|---|---|---|
| NPU (32 chips) | 76,292 | 0.8067 | 0.7993 | 1.1531 |
| B200 GPU | 76,293 | 0.8070 | 0.7994 | 1.1533 |
On the unseen text, the paired difference over identical windows is -0.0001 (NPU minus GPU) with a standard error of 0.0007, and the NPU model is lower on 50.1% of windows.
The differences (0.0003 and 0.0001) are far inside measurement noise. The NPU reached GPU quality.
The B200 GPU weights are included in this repository under b200_reference/, so anyone can compare the two runs directly:
python
from transformers import AutoModelForCausalLMnpu = AutoModelForCausalLM.from_pretrained("FINAL-Bench/VKORE-325M-NPU-Base")gpu = AutoModelForCausalLM.from_pretrained("FINAL-Bench/VKORE-325M-NPU-Base", subfolder="b200_reference")| Item | Value |
|---|---|
| Parameters | 325.6M |
| Architecture | Decoder only Transformer (RoPE, RMSNorm, SwiGLU, GQA) |
| Layers | 20 |
| Hidden width | 1024 |
| FFN width | 3584 |
| Heads | 8 (KV 2), head dim 256 |
| Vocabulary | 256 (byte level) |
| Context length | 256 |
| Training data | FineWeb-Edu sample-10BT (public dataset) |
| Training tokens | 5.0B |
| Training loss | 5.73 to 0.80 |
| Evaluation loss | 0.8067 |
sample/10BT (public)Prompt: Photosynthesis is the process
Output (sampling, temperature 0.7):
Photosynthesis is the process by which plants convert sunlight into energy.
python
import torchfrom transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("FINAL-Bench/VKORE-325M-NPU-Base")prompt = "Photosynthesis is the process"ids = torch.tensor([list(prompt.encode("utf-8"))]) # byte level: token id = byte valueout = model.generate(ids, max_new_tokens=200, do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.15)print(bytes(out[0].tolist()).decode("utf-8", errors="replace"))A byte level tokenizer is used (byte values 0 to 255), so no tokenizer file is needed.
This is a base model (no instruction tuning) released as a research demonstration of from scratch pretraining on an inference only NPU.
VIDRAFT. Part of the VKORE project (NPU acceleration family). Published under FINAL-Bench.
from transformers import AutoModelForCausalLMnpu = AutoModelForCausalLM.from_pretrained("FINAL-Bench/VKORE-325M-NPU-Base")gpu = AutoModelForCausalLM.from_pretrained("FINAL-Bench/VKORE-325M-NPU-Base", subfolder="b200_reference")import torchfrom transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("FINAL-Bench/VKORE-325M-NPU-Base")prompt = "Photosynthesis is the process"ids = torch.tensor([list(prompt.encode("utf-8"))]) # byte level: token id = byte valueout = model.generate(ids, max_new_tokens=200, do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.15)print(bytes(out[0].tolist()).decode("utf-8", errors="replace"))