GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
nkthebass
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
nkthebass
Model Tree
Base
this model
Input Modalities
Text
Output Modalities
Text
Supported Functionality
Dedicated Endpoints
Container
| Parameters | ~303M |
| Architecture | LLaMA-style (LlamaForCausalLM) — RoPE, RMSNorm, SwiGLU, GQA |
| Layers / hidden / heads | 24 / 1024 / 16 (4 KV heads) |
| FFN / vocab / context | 2816 / 32,000 / 1024 |
| Tied embeddings | Yes |
| Special tokens | <|user|> <|assistant|> <|system|> <|end|> |
| EOS token | <|end|> |
This is the single most important detail. The tokenizer normalizes newlines to spaces, so the model was trained with spaces between turns. The bundled chat_template already does this — use apply_chat_template and it just works:
python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torchtok = AutoTokenizer.from_pretrained("your-username/tinybrainbot-303m-instruct")model = AutoModelForCausalLM.from_pretrained("your-username/tinybrainbot-303m-instruct", torch_dtype=torch.float16).eval()msgs = [{"role": "user", "content": "What is the capital of France?"}]prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)ids = tok(prompt, return_tensors="pt", add_special_tokens=False).input_idsout = model.generate(ids, max_new_tokens=64, do_sample=True,temperature=0.5, top_p=0.9, repetition_penalty=1.2, eos_token_id=7)print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))# -> Paris.
The rendered format is (note the spaces, no \n):
markdown
<|user|> {message} <|end|> <|assistant|>
If you build prompts by hand or configure a UI (llama.cpp / Ollama / LM Studio / Jan), make sure the separators are spaces and the stop token is <|end|> — a template with literal \n will feed the model out-of-distribution tokens and it will ramble without stopping.
Small models need a lower temperature than large ones (temp 1.0 makes this model incoherent).
| Use case | temp | top-p | rep penalty |
|---|---|---|---|
| General chat (default) | 0.7 | 0.9 | 1.2 |
| More creative | 0.8 | 0.9 | 1.2 |
| Factual / reliable | 0.4–0.6 | 0.9 | 1.15 |
Standard multiple-choice benchmarks (log-likelihood scoring, n=200 subsample — absolute numbers run a bit low vs. the official lm-eval-harness; use them for relative comparison):
| Benchmark | TinyBrainBot 303M | random |
|---|---|---|
| ARC-Easy (acc_norm) | ~44% | 25 |
| OpenBookQA (acc_norm) | ~31% | 25 |
| HellaSwag (acc_norm) | ~29% | 25 |
| WinoGrande (acc) | ~46% | 50 |
| ARC-Challenge (acc_norm) | ~24% | 25 |
| MMLU (acc) | ~23% |
Roughly GPT-2-small tier on general reasoning, with a relative edge on knowledge QA (ARC / OpenBookQA) from the fact distillation. Near-random on MMLU, as expected at this scale.
(best of 5 samples, temperature tuned per prompt):
Q: What is the largest planet in the solar system? Jupiter.
Q: Explain what photosynthesis is in one or two simple sentences. Photosynthesis is how plants use sunlight, water, and air to make food (glucose) and release oxygen. It occurs in many types like green plants, algae, and trees.
Q: Write a short poem about the moon. Silver moon, slow and gold, hums low over the sleeping sky. It dreams in silver rivers, guarding dreams in morning light.
Q: List three healthy breakfast ideas. Greek yogurt with berries, whole wheat bread, cream cheese.
Q: Give me one tip for staying focused while studying. Use 25-minute focus blocks with 5-minute breaks, silence notifications, and a clear workspace to keep your mind fresh.
Apache-2.0 (change if you prefer).
| 25 |