GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nkthebass
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
nkthebass
Model Tree
Input Modalities
Output Modalities
Supported Functionality
It is a standard Mixtral model, so it needs no custom code:
python
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "nkthebass/tinybrainbot-61m-a10m-v4.1-moe-base"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo) ids = tok("Once upon a time, a little fox", return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=80) # defaults from generation_config.jsonprint(tok.decode(out[0], skip_special_tokens=True))GGUF (llama.cpp, LM Studio, Ollama): tinybrainbot-61m-a10m-v4.1-moe-base-F16.gguf (129 MB). It
tokenizes and generates identically to the transformers version.
text
llama-completion -m tinybrainbot-61m-a10m-v4.1-moe-base-F16.gguf -no-cnv -p "Once upon a time, a little fox" -n 80 --temp 0.7 --repeat-penalty 1.15Default sampling: temperature 0.7, top_p 0.95, top_k 40, repetition_penalty 1.15.
Unedited, at the default settings.
Once upon a time, a little fox named Sammy had an idea. He wanted to create an amazing treehouse made of pebbles and stones so that it could be easily accessed by everyone.
Photosynthesis is the process by which plants absorb light energy and convert it into chemical energy.
The best way to learn a new language is to try it out. To learn a new language, you need to know the words and phrases that make up the language.
| Architecture | Mixtral-style sparse MoE decoder: RoPE, RMSNorm, grouped-query attention, SwiGLU experts |
| Shape | hidden 256, 14 layers, 4 heads of 64, 2 KV heads; 16 experts of width 336 per layer, top-2 routing |
| Parameters | 60.62M non-embedding total, 10.04M active per token, + 3.58M tied embedding |
| Context | 1,024 tokens |
| Vocabulary | 14,000 (the v4 byte-level BPE: digits split, newlines kept, no unknown tokens) |
Attention is identical to the dense 10M; only the feed-forward layer changes. Its width (672) is split into experts of half that width, and each token uses two, so a token costs exactly what it costs in the dense model while the layer stores 8x the feed-forward parameters. Routing is Mixtral's (softmax over the experts, keep the top 2, renormalise their weights), trained with a 0.01 load-balancing loss, which keeps all 16 experts in use: in a typical training batch the least-used expert still gets about half of its fair share of tokens.
11.0B tokens on 2x Tesla V100 16GB, about 25.5 hours, fp16, AdamW (0.9, 0.95), weight decay 0.1, peak LR 3e-3, 131k tokens per step, warmup-stable-decay - the v4 scaling study's exact data and schedule. The expert layers run on the sigma-MoE Triton kernel (Csordas et al.), which fuses token routing into the matrix multiply.
| Group | Share | Sources |
|---|---|---|
| Web | 37% | FineWeb-Edu 28, DCLM 9 |
| Reference | 13.5% | Wikipedia article leads 12, full Wikipedia 1.5 |
| Narrative | 11% | Project Gutenberg 8, BookCorpusOpen 3 |
| Code | 9% | The Stack (smol) 7.5, Python/JS 1.5 |
| Math | 9% | MegaMath-Web-Pro 6, FineMath-4+ 3 |
| Synthetic textbooks | 7.5% |
Then 500M tokens of the same mix plus 7.8% generated everyday-commonsense, procedural, worked arithmetic and short-fact writing (no benchmark data), and an 803M-token fact-dense anneal while the learning rate decays to zero.
text
lm_eval --model hf --model_args pretrained=nkthebass/tinybrainbot-61m-a10m-v4.1-moe-base,dtype=float32 \ --tasks arc_easy,arc_challenge,sciq,openbookqa,mmlu,hellaswag,piqa,winogrande,social_iqa,commonsense_qa,lambada_openai,boolq,mathqa \ --num_fewshot 0 --limit 2000 --batch_size 8 --seed 1234Dense comparison models: TinyBrainBot-25M-v4-Base and the v4 scaling study (500K to 25M).
llama-completion -m tinybrainbot-61m-a10m-v4.1-moe-base-F16.gguf -no-cnv -p "Once upon a time, a little fox" -n 80 --temp 0.7 --repeat-penalty 1.15lm_eval --model hf --model_args pretrained=nkthebass/tinybrainbot-61m-a10m-v4.1-moe-base,dtype=float32 \ --tasks arc_easy,arc_challenge,sciq,openbookqa,mmlu,hellaswag,piqa,winogrande,social_iqa,commonsense_qa,lambada_openai,boolq,mathqa \ --num_fewshot 0 --limit 2000 --batch_size 8 --seed 1234| Cosmopedia (Stanford, OpenStax, auto-math) |
| Q&A and facts | 7% | distilled Q&A, short facts, cause-and-effect explanations |
| Educational PDFs | 5.5% | FinePDFs-Edu |
| Reasoning | 0.5% | distilled short reasoning |