GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nkthebass
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
nkthebass
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Zero-shot, full test sets, acc_norm; (raw acc) underneath. Intelligence Index from the Open SLM Leaderboard.
| Model | Params | Int Index | HellaSwag | ARC-E | ARC-C | PIQA | ArithMark-3 |
|---|---|---|---|---|---|---|---|
| TinyBrainBot-360M-A75M-v5.1-Base (this model) | 383M (75M active) | 29.76 | 38.68 | 58.96 | 36.18 | 68.01 | 53.00 |
| (raw acc) | 27.93 | 32.75 | 64.60 | 33.36 | 67.68 | 53.00 | |
| TinyBrainBot-360M-A75M-v4.1-MoE-Base (previous) | 383M (75M active) | 24.40 | 37.27 | 55.64 | 32.34 | 66.32 | 42.00 |
| GPT-X3-150M | 149M | 33.05 | 46.2 | 59.6 | 31.3 | 71.3 | 50.9 |
| Boris-2-Preview-2 | 144M | 30.58 | 46.6 | 59.1 | 30.2 | 70.8 | 42.2 |
| KLM-1.5 | 99M | 30.26 | 43.7 | 59.7 | 32.7 | 68.5 | 48.2 |
| Sol Pro | 138M | 28.60 | 46.7 | 54.0 | 31.8 | 71.0 | 36.0 |
Other models' numbers are from the leaderboard.
On 1,380 held-out arithmetic problems in topics and wordings the model never saw, it scores 47.3, close to its public ArithMark-3 of 53.0.
python
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v5.1-base")model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v5.1-base")ids = tok("The water cycle begins when the sun heats water in oceans and lakes, causing it to", return_tensors="pt")print(tok.decode(model.generate(**ids, max_new_tokens=30, do_sample=False, repetition_penalty=1.0)[0]))GGUF files for llama.cpp, LM Studio, Ollama and Jan: F16 (768 MB), Q8_0 (409 MB) and Q4_K_M (239 MB). Greedy output matches transformers token for token.
This is a base model: it continues text and does not chat. Its chat version, TinyBrainBot-360M-A75M-v5.1-Instruct, is in the TinyBrainBot v5.1 collection.
| Parameters | 383.4M total; 75.7M active non-embedding per token + 24.6M tied embedding |
| Shape | hidden 768, 12 layers, 12 query heads of 64 with 4 KV heads (GQA), RMSNorm, RoPE |
| Experts | 24 SwiGLU experts of width 512 per layer, top-4 routing |
| Vocabulary | 32,000 SentencePiece; line breaks are kept in this version (earlier TinyBrainBot models on this tokenizer never saw them) |
| Context | 2,048 tokens |
17.9B tokens: 16.7B of pretraining at constant learning rate, then a new 1.2B-token decay.
| Pretraining (16.7B) | Share |
|---|---|
| Educational web: FineWeb-Edu, DCLM, finepdfs-edu | 44% |
| Wikipedia | 13% |
| Books and stories: Gutenberg, BookCorpus, TinyStories | 11% |
| Math: MegaMath, FineMath, Orca-Math | 9% |
| Code: the-stack-smol | 9% |
| Cosmopedia (Stanford, OpenStax, math) | 7% |
| Q&A, facts and reasoning (our distilled sets) | 7% |
| Decay (1.2B) | Share |
|---|---|
| Q&A and facts: our distilled sets, GooAQ, FactBase-DCLM, finephrase FAQ, ELI5 | 26% |
| School-level textbooks: SmolLM cosmopedia-v2 and Cosmopedia, school audiences | 22% |
| Educational and broad web: top-scored FineWeb-Edu pages, DCLM-Ultra-1.1, ClimbMix, WikiHow | 22% |
| Wikipedia (dense and Simple English) | 12% |
| Grade-school science Q&A | 10% |
| Commonsense, how-to and writing (our distilled sets) | 5% |
| Arithmetic (our drills, Orca-Math, UltraData-Math) | 3% |
The grade-school science Q&A (about 420k questions with short explained answers, no answer options) was written by Ling-3.0-Flash; any item sharing an 8-word phrase with an ARC question was removed before training. No benchmark train or test data was used.
Recipe: AdamW, peak LR 8e-4 with a warmup-stable-decay schedule, 262k tokens per step, fp16, 2x V100.
text
lm_eval --model hf --model_args pretrained=nkthebass/tinybrainbot-360m-a75m-v5.1-base,dtype=float32 --tasks hellaswag,arc_easy,arc_challenge,piqa --num_fewshot 0python bencharithmark-3.py --model nkthebass/tinybrainbot-360m-a75m-v5.1-base --dtype float32A small base model: good at short factual and commonsense continuations for its size, but open-ended generation is limited and can be wrong. Not aligned or safety-tuned.
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v5.1-base")model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v5.1-base")ids = tok("The water cycle begins when the sun heats water in oceans and lakes, causing it to", return_tensors="pt")print(tok.decode(model.generate(**ids, max_new_tokens=30, do_sample=False, repetition_penalty=1.0)[0]))lm_eval --model hf --model_args pretrained=nkthebass/tinybrainbot-360m-a75m-v5.1-base,dtype=float32 --tasks hellaswag,arc_easy,arc_challenge,piqa --num_fewshot 0python bencharithmark-3.py --model nkthebass/tinybrainbot-360m-a75m-v5.1-base --dtype float32