GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nkthebass
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
nkthebass
Model Tree
Input Modalities
Output Modalities
Supported Functionality
It is a standard Mixtral model and needs no custom code:
python
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "nkthebass/tinybrainbot-360m-a75m-v4.1-moe-base"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo) ids = tok("Once upon a time, a little fox", return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=80) # defaults from generation_config.jsonprint(tok.decode(out[0], skip_special_tokens=True))GGUF (llama.cpp, LM Studio, Ollama): tinybrainbot-360m-a75m-v4.1-moe-base-Q8_0.gguf (409 MB) or
-F16.gguf (768 MB). Both tokenize exactly like the transformers version and give the same
greedy output.
text
llama-completion -m tinybrainbot-360m-a75m-v4.1-moe-base-Q8_0.gguf -no-cnv -p "Once upon a time, a little fox" -n 80 --temp 0.7 --repeat-penalty 1.15This is a base model: it continues text, it does not follow chat instructions. Default sampling:
temperature 0.7, top_p 0.9, repetition_penalty 1.15. A chat version is in progress.
Unedited, at the default settings.
Once upon a time, a little fox named Max lived in a small village. One day, Max heard his name mentioned by someone else. He asked around the town and found that everyone had lost their way in the forest. But then, he noticed something strange happening - some trees were open wide! All of the animals were moving quickly toward each other because they wanted to find food and shelter. The fox realized that the path wasn't clear at all; instead, it seemed like it
Question: Why do we have seasons on Earth? Answer: The tilt of Earth's axis causes changing sunlight angles.
The best way to learn a new language is by engaging with it. This means watching movies, reading books, attending cultural events, and practicing speaking the language regularly. It's not just about memorizing words; it's also about understanding grammar rules and vocabulary lists. By doing so, you will gain confidence in communicating effectively and expressing yourself in the target language. Another important aspect of learning a new language is improving your listening skills. Listening to native speakers can help you understand conversations and
| Benchmark (0-shot) | this model | before the expert patch |
|---|---|---|
| ARC-Easy | 55.9 | 54.2 |
| ARC-Easy (raw acc) | 59.8 | 59.2 |
| ARC-Challenge | 32.3 | 31.9 |
| ARC-Challenge (raw acc) | 28.8 | 27.8 |
| SciQ | 73.3 | 72.5 |
| SciQ (raw acc) |
lm-eval-harness, 0-shot, up to 2,000 examples per task (HellaSwag and MMLU use their first 2,000;
the leaderboard table above uses full sets). Each task's first row is acc_norm where the task
reports it, acc otherwise; the "(raw acc)" row under it is acc. "Before the expert patch" is the
same model at the end of pretraining.
| Architecture | Mixtral-style sparse MoE decoder: RoPE, RMSNorm, grouped-query attention, SwiGLU experts |
| Shape | hidden 768, 12 layers, 12 heads of 64, 4 KV heads; 24 experts of width 512 per layer, top-4 routing |
| Parameters | 358.9M non-embedding total, 75.7M active per token, + 24.6M tied embedding (383.4M total) |
| Context | 2,048 tokens |
| Vocabulary | 32,000 (SentencePiece, byte fallback) |
Routing is Mixtral's: softmax over the 24 experts, keep the top 4, renormalise their weights. A 0.01 load-balancing loss keeps every expert in use. The experts run on the sigma-MoE Triton kernel (Csordas et al.), which fuses token routing into the matrix multiply, so training ran at about 30k tokens/s on the two V100s.
Pretraining: 16.7B tokens, fp16, AdamW (0.9, 0.95), weight decay 0.1, peak LR 8e-4, 262k tokens per step, warmup-stable-decay.
| Group | Share | Sources |
|---|---|---|
| Web | 35% | FineWeb-Edu 27, DCLM 9 |
| Reference | 14% | Wikipedia article leads 12, full Wikipedia 1.5 |
| Math | 11% | MegaMath-Web-Pro 5, FineMath-4+ 4, Cosmopedia auto-math 1, Orca-Math 1 |
| Narrative | 10% | Project Gutenberg 4, TinyStories 4, BookCorpusOpen 3 |
| Code | 8% | The Stack (smol) 7, Python/JS 1.5 |
| Synthetic textbooks |
Anneal: 1.5B tokens while the learning rate decays to zero: mostly distilled Q&A and short facts, with explanations and short reasoning, Wikipedia, Cosmopedia (including WikiHow), FineMath, FinePDFs and short generated everyday scenes and stories.
Expert patch: factual Q&A. In each layer, the 4 experts that factual question-answer text uses most (relative to ordinary text) were trained for 131M tokens on plain-text Q&A, short facts and explanations, mixed with pretraining text. Everything else stayed frozen - attention, norms, embeddings, router and the other 20 experts per layer - and was checked bit-for-bit after every step. It is one model fine-tuned in place, not a merge. The patch raised ARC-Easy and ARC-Challenge without moving the other benchmarks (see "All benchmarks").
text
lm_eval --model hf --model_args pretrained=nkthebass/tinybrainbot-360m-a75m-v4.1-moe-base,dtype=float32 \ --tasks hellaswag,arc_easy,arc_challenge,piqa --num_fewshot 0 --batch_size 16| 81.8 |
| 82.1 |
| OpenBookQA | 34.4 | 33.8 |
| OpenBookQA (raw acc) | 23.6 | 23.2 |
| MMLU | 24.2 | 24.4 |
| HellaSwag | 41.5 | 41.9 |
| HellaSwag (raw acc) | 35.8 | 35.4 |
| PIQA | 66.3 | 66.8 |
| PIQA (raw acc) | 66.5 | 66.6 |
| WinoGrande | 52.4 | 52.6 |
| Social IQa | 39.7 | 39.1 |
| CommonsenseQA | 21.6 | 20.9 |
| LAMBADA (OpenAI) | 28.6 | 28.5 |
| BoolQ | 57.8 | 58.6 |
| MathQA | 23.2 | 23.9 |
| MathQA (raw acc) | 24.1 | 24.7 |
| Average (13) | 42.4 | 42.2 |
| LAMBADA perplexity (lower is better) | 65.7 | 64.2 |
| 8% |
| Cosmopedia (Stanford, OpenStax) |
| Q&A and facts | 7% | distilled Q&A, short facts, cause-and-effect explanations, short reasoning |
| Educational PDFs | 7% | FinePDFs-Edu |