GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nkthebass
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
nkthebass
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Zero-shot, full test sets, acc_norm. The (raw acc) row shows plain accuracy. The Intelligence Index is the Open SLM Leaderboard's; the other models' numbers come from that leaderboard.
| Model | Params | Int Index | HellaSwag | ARC-E | ARC-C | PIQA | ArithMark-3 |
|---|---|---|---|---|---|---|---|
| TinyBrainBot-3M-v2-Base (this model) | 2.94M | 10.38 | 27.29 | 33.38 | 22.53 | 56.75 | 45.10 |
| (raw acc) | 10.61 | 26.83 | 33.67 | 18.69 | 58.65 | 45.40 | |
| GPT-S3-3M | 2.97M | 9.61 | 27.68 | 34.76 | 23.12 | 57.02 | 39.10 |
| Tokle-3M | 2.91M | 8.91 | 27.20 | 34.85 | 23.98 | 55.01 | 40.80 |
| pulvis-v2 | 2.96M | 8.62 | 27.93 | 31.86 | 22.78 | 56.86 | 37.40 |
| Purrence-3M | 2.99M | 8.49 | 27.81 | 34.55 | 23.63 | 56.64 | 34.80 |
The arithmetic share was tuned so ArithMark-3 stays close to how the model does on problems written by other people. None of these sets was used in training (Gemma and Llama were never used to write training data), all acc_norm:
| Held-out set | Score |
|---|---|
| ArithMark-3 (public, for reference) | 45.1 |
| Arithmetic stories written by Gemma (544) | 34.2 |
| Arithmetic stories written by Llama (1,093) | 36.3 |
| 23 arithmetic topics ArithMark-3 lacks, new wordings (1,380) | 38.8 |
| Unknown positions, distractors, tables, layouts (480) | 37.1 |
| Held-out number formats: large / commas / money | 40.8 / 46.7 / 45.2 |
| Bare equations from ArithMark-3 ("43 + 19 =") | 35.9 |
| ArithMark-2 | 27.2 |
python
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-3m-v2-base")model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-3m-v2-base")ids = tok("To keep a cut apple from turning brown, you can", return_tensors="pt")print(tok.decode(model.generate(**ids, max_new_tokens=20, do_sample=False)[0]))There is also an F16 GGUF (tinybrainbot-3m-v2-base-F16.gguf, 6 MB) for llama.cpp, LM Studio and Jan. A Q8_0 file isn't possible because the
hidden size (144) is not a multiple of 32.
This is a base model: it continues text and does not chat. At this size expect short, simple text.
| Parameters | 2,942,928 total (2.35M non-embedding + 0.59M tied embedding) |
| Shape | hidden 144, 10 layers, 3 query heads of 48 with 1 KV head (GQA), SwiGLU 416, RMSNorm, RoPE |
| Vocabulary | 4,096-token byte-level BPE trained for this model: digits split one per token, whitespace kept exactly |
| Context | 1,024 tokens |
6.05B tokens in three parts: the first 4.72B are TinyBrainBot-3M-Base's own pretraining (its step 36,000), then 0.81B of new pretraining and a 0.51B decay.
| New pretraining (0.81B) | Share |
|---|---|
| Q&A and facts (our distilled plain-text sets) | 25% |
| Arithmetic in many forms (below) | 20% |
| Everyday / how-to / commonsense (Cosmopedia WikiHow + distilled) | 15% |
| Cosmopedia Stanford + OpenStax | 13% |
| TinyStories | 12% |
| FineWeb-Edu | 10% |
| Wikipedia leads | 5% |
| Decay (0.51B) | Share |
|---|---|
| Q&A and facts | 34.3% |
| Everyday / how-to / commonsense | 22% |
| Cosmopedia Stanford | 15% |
| Wikipedia leads | 10% |
| TinyStories | 10% |
| Arithmetic | 8.7% |
The arithmetic mixes several sources so the skill is not tied to one wording: drills generated by our own programs (order of operations, negatives, number formats, number words, whitespace variants, tables, unknown positions, distractors; every answer computed in Python), story problems in sentence frames written by several LLM families (Ling, Mistral; the numbers filled in and computed by Python), and Orca-Math word problems. No benchmark training or test data was used.
Recipe: AdamW (betas 0.9/0.95, weight decay 0.1), LR 2.7e-3 held then decayed to 0 (warmup-stable-decay), 131k tokens per step, fp16 mixed precision, 2x V100.
How this checkpoint was chosen. Several decay branches with different arithmetic shares were run from the same checkpoint (step 42,212). A little story-style arithmetic raised ArithMark-3 much faster than any held-out set, so we published the branch whose public ArithMark-3 stays within about 10 points of the held-out sets, not the one with the highest Index (13.93, but ArithMark-3 62.8 against 41.0 held-out). The published weights are that branch's final checkpoint, not a merge.
Float32, lm-eval 0.4.13 (full sets, 0-shot) and the official ArithMark-3 script:
text
lm_eval --model hf --model_args pretrained=nkthebass/tinybrainbot-3m-v2-base,dtype=float32 --tasks hellaswag,arc_easy,arc_challenge,piqa --num_fewshot 0python bencharithmark-3.py --model nkthebass/tinybrainbot-3m-v2-base --dtype float32At 3M parameters this is a toy-scale model. It knows few facts, loses coherence after a few sentences, and its arithmetic is unreliable outside simple one-step problems. HellaSwag and ARC are close to chance for every model this size, so the gaps between models there are small.
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-3m-v2-base")model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-3m-v2-base")ids = tok("To keep a cut apple from turning brown, you can", return_tensors="pt")print(tok.decode(model.generate(**ids, max_new_tokens=20, do_sample=False)[0]))lm_eval --model hf --model_args pretrained=nkthebass/tinybrainbot-3m-v2-base,dtype=float32 --tasks hellaswag,arc_easy,arc_challenge,piqa --num_fewshot 0python bencharithmark-3.py --model nkthebass/tinybrainbot-3m-v2-base --dtype float32