GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nkthebass
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
nkthebass
Model Tree
Input Modalities
Output Modalities
Supported Functionality
python
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "nkthebass/tinybrainbot-25m-v4-base"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo) ids = tok("Once upon a time, a little fox", return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=80) # defaults from generation_config.jsonprint(tok.decode(out[0], skip_special_tokens=True))GGUF (llama.cpp, LM Studio, Ollama): tinybrainbot-25m-v4-base-F16.gguf (59 MB) and
tinybrainbot-25m-v4-base-Q8_0.gguf (31 MB) are in this repo. They tokenize and generate
identically to the transformers version.
text
llama-cli -m tinybrainbot-25m-v4-base-Q8_0.gguf -p "Once upon a time, a little fox" -n 80 --temp 0.7 --repeat-penalty 1.15Default sampling: temperature 0.7, top_p 0.95, top_k 40, repetition_penalty 1.15.
Unedited, at the default settings.
Once upon a time, a little fox named Mia decided to start her own small business selling handmade crafts at the local community center. She wanted to make sure that everyone had enough supplies and opportunities to buy their favorite items. So, she gathered some friends and helped them find a creative idea for making their own goods.
Dear diary, today was a fun and exciting day with lots of memories! I found out that every single day, we learned about the fascinating world of animal behavior and how it affects us all. So let's continue exploring this amazing subject together!
Photosynthesis is the process by which light energy is converted into chemical energy (e.g., glycolysis). Photosynthetic bacteria convert sunlight, carbon dioxide, and water to glucose (a sugar-sweetened food), oxygen, and nitrogen, respectively.
| Architecture | Llama-compatible decoder, RoPE, SwiGLU, RMSNorm, grouped-query attention |
| Shape | hidden 384, 16 layers, 6 heads of 64, 2 KV heads, FFN 1024 |
| Parameters | 25.18M non-embedding + 5.38M tied embedding = 30.55M |
| Context | 1,024 tokens |
| Vocabulary | 14,000 |
Deep and narrow on purpose: at this size, adding layers helps more than adding width.
A new 14k byte-level BPE trained for this model on a sample of its own training mix:
<|user|>, <|assistant|>, <|end|>, <think>, </think>) are
single tokens, reserved for fine-tuning. <|endoftext|> (id 0) separates documents.11.0B tokens on 2x Tesla V100 16GB, about 22 hours, fp16, AdamW (0.9, 0.95), weight decay 0.1, peak LR 2e-3, 131k tokens per step, warmup-stable-decay schedule.
Stage 1 - 9.7B tokens, general mix. Every source under 3 epochs.
| Group | Share | Sources |
|---|---|---|
| Web | 37% | FineWeb-Edu 28, DCLM 9 |
| Reference | 13.5% | Wikipedia article leads 12, full Wikipedia 1.5 |
| Narrative | 11% | Project Gutenberg 8, BookCorpusOpen 3 |
| Code | 9% | The Stack (smol) 7.5, Python/JS 1.5 |
| Math | 9% | MegaMath-Web-Pro 6, FineMath-4+ 3 |
| Synthetic textbooks | 7.5% | Cosmopedia (Stanford, OpenStax, auto-math) |
| Q&A and facts | 7% | distilled Q&A, short facts, cause-and-effect explanations |
| Educational PDFs | 5.5% | FinePDFs-Edu |
| Reasoning | 0.5% | distilled short reasoning |
Stage 2 - 500M tokens. The same mix with 7.8% added: everyday physical-commonsense writing (2.5%), procedural "goal and method" writing (1.8%), worked order-of-operations problems (2%) and short facts (+1.5%). All of it is generated text written for this purpose; none of it is benchmark data. Stage 1 ended with a short LR decay, which stage 2 re-warmed.
Stage 3 - 803M token anneal. LR decays to zero over a fact-dense mix: Cosmopedia, FineMath, FinePDFs-Edu, Wikipedia leads, distilled Q&A and reasoning, plus the stage 2 additions.
The packed corpus was block-shuffled and its document density checked shard by shard before training, so every stage is an even mix rather than one source after another.
text
lm_eval --model hf --model_args pretrained=nkthebass/tinybrainbot-25m-v4-base,dtype=float32 \ --tasks arc_easy,arc_challenge,sciq,openbookqa,mmlu,hellaswag,piqa,winogrande,social_iqa,commonsense_qa,lambada_openai,boolq,mathqa \ --num_fewshot 0 --limit 2000 --batch_size 8 --seed 1234from transformers import AutoModelForCausalLM, AutoTokenizer repo = "nkthebass/tinybrainbot-25m-v4-base"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo) ids = tok("Once upon a time, a little fox", return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=80) # defaults from generation_config.jsonprint(tok.decode(out[0], skip_special_tokens=True))lm_eval --model hf --model_args pretrained=nkthebass/tinybrainbot-25m-v4-base,dtype=float32 \ --tasks arc_easy,arc_challenge,sciq,openbookqa,mmlu,hellaswag,piqa,winogrande,social_iqa,commonsense_qa,lambada_openai,boolq,mathqa \ --num_fewshot 0 --limit 2000 --batch_size 8 --seed 1234