nkthebass

tinybrainbot-320mV2-math

README

License: apache-2.0

What it does well

Table with columns: Skill, Method, Result
Skill	Method	Result
Multi-digit add / subtract (2–10 digit, comma-formatted)	column-by-column with carries/borrows	~90–100%
Word problems (large numbers, multi-step, mixed verbs)	reads the problem → delegates to column / partial-product computation	solves the full target set
2-digit multiplication	partial products + column addition	~88%
Division	long division	reliable on simple cases
Greetings / short answers	—	fine

It reads the problem and computes — e.g. "A store had 56,321 items and sold 28,479. How many remain?" →

markdown
<think> Start with 56321. Then subtract 28479. Subtract column by column:
ones: 11 - 9 = 2, borrow 1. ... So 56321 - 28479 = 27842. </think>
The answer is 27842.

Evaluation

GPT-3 Arithmetic protocol (exact-match) — vs GPT-3-175B (few-shot, direct):

Table with columns: Task, GPT-3 175B, This model
Task	GPT-3 175B	This model
2-digit add	~100%	100%
2-digit sub	~99%	95%
3-digit add	80.4%	100%
3-digit sub	94.2%	95%
4-digit add	25.5%	100%
4-digit sub	26.8%

Ours uses trained-in worked steps; GPT-3's numbers are direct-answer. Both are pure LMs with no external tools/calculators. The point is about method: teaching a 326M model the algorithm beats a 175B model guessing — decisively on 4–5-digit arithmetic.

Word-problem set (large-number add/sub with commas, multi-step, 2-digit multiply, first-person phrasings): solves essentially all of a 20-problem targeted set by reading the problem and computing the steps.
GSM8K: ~3–4% (zero-shot CoT, n=500) — off the base instruct's 0.53% floor, at roughly the SmolLM2-360M-Instruct tier. Arbitrary hard multi-step word problems remain scale-limited at 326M.

General benchmarks (log-likelihood MC, our harness; the math SFT did not erode general ability):

Table with columns: HellaSwag, ARC-Easy, ARC-Challenge, OpenBookQA, WinoGrande, MMLU
HellaSwag	ARC-Easy	ARC-Challenge	OpenBookQA	WinoGrande	MMLU
35.0	49.2	30.5	32.0	54.9	27.3

Reaches the Pythia-410M tier — a model trained on ~30× more tokens — while being math-specialized.

Usage

Chat format:

markdown
<|user|>
{question}
<|end|>
<|assistant|>
{answer}
<|end|>

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tok = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-320mV2-math")
m = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-320mV2-math", torch_dtype=torch.float16)
ids = tok.apply_chat_template([{"role":"user","content":"A theater has 56 rows with 27 seats in each row. How many seats?"}],
                              add_generation_prompt=True, return_tensors="pt")
print(tok.decode(m.generate(ids, max_new_tokens=256, do_sample=False)[0][ids.shape[1]:], skip_special_tokens=True))

GGUF file (*-F16.gguf) works directly in LM Studio / Ollama / llama.cpp.

Prompting tips

This is a math model — it is strongest on arithmetic and worked-step word problems, and its general-knowledge chat is weak (a 326M capacity limit). For non-math questions, ask direct, specific questions rather than open-ended imperatives; short or empty replies on some phrasings (and in long multi-turn chats) are a model-capacity characteristic, not a GGUF/format bug. The F16 GGUF is a faithful conversion — token-for-token identical to the fp16 transformers model (verified). Use the built-in chat template as-is; spaces between the role tags are correct (SentencePiece normalizes newlines to spaces, so both forms encode to the same token IDs the model trained on).

Model details

Table

Parameters	~325.9M (1024 hidden · 26 layers · 16h / 4kv GQA · ffn 2816 · ctx 1024)
Vocab / tokenizer	32,000 · `tbb-32k-v2` (BPE)
Precision	fp16
Training	from-scratch pretrain (~10B tokens, WSD) → math/reasoning SFT (assistant-masked, chat format). Arithmetic taught as explicit worked steps.

Training process

$training loss curve$

Pretraining — from scratch, 51,000 steps / ~10.03B tokens on 2× Tesla V100 (PyTorch DDP gloo, fp16 + GradScaler, fused AdamW). Warmup–Stable–Decay schedule: 1,000-step warmup → stable LR 6e-4 → cosine decay over the final ~20% (from step 40,800). A quality-anneal (swap to a knowledge-dense data mix) runs over the last ~3B tokens — the visible dip near step 40k. 13-source data mix, principle real > synthetic (≤ ~35%): DCLM web, Wikipedia leads, FineWeb-edu, filtered Python/JS code, verified arithmetic, and distilled Q&A/facts/reasoning. Pretrain loss ~10.6 → ~2.3.
Math-reasoning SFT (steps 51k → 58k, green) — supervised fine-tuning (assistant-masked, chat format) that teaches: multi-digit arithmetic as explicit worked steps (column add/sub, long division, partial-product multiply); word problems that read the problem then delegate the arithmetic to column computation (large numbers, commas, multi-step, first-person phrasings); plus retained general chat / greetings. The data was iteratively refined to kill template-overfit (phantom steps), cover diverse verbs and first-person forms, and handle large/comma-formatted numbers. SFT loss → ~0.4.

Limitations

General knowledge is weak — it can drift into confident errors on factual/open-ended questions. This is a fundamental 326M capacity limit, not a bug. Use it for math, not facts.
Novel word-problem phrasings can still trip it (it may drop a step on unusual structures).
Hard multi-step reasoning (GSM8K/MATH) caps at this scale.
3+ digit multiplication and large-number division are soft spots.
English only, 1024-token context, no RLHF/safety tuning — outputs may be wrong or inappropriate; don't rely on them unchecked.

Hardware & framework

2× NVIDIA Tesla V100-PCIE-16GB · Windows · PyTorch DDP (gloo) · fp16 · custom TinyBrainBot trainer.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

nkthebass

Model Tree

Base

nkthebass/tinybrainbot-320mV2-base

Quantized

this model

Input Modalities

Text

Output Modalities