GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
SmallAICreator
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
SmallAICreator
Model Tree
SmallAICreator/AuroraGPT-Qwen-Distill
Input Modalities
Output Modalities
Supported Functionality
Measured, not estimated:
| benchmark | score |
|---|---|
| GSM8K (40 problems, chat format, greedy) | 1/40 = 2.5% |
| GSM8K — the base model it was trained from | 0/40 = 0.0% |
It cannot do GSM8K. 2.5% vs 0.0% on 40 problems is inside the noise (±7.7pp); the honest statement is that neither this model nor its base can solve grade-school word problems reliably. On a hand-picked 5-problem probe set it scores 3/5, but those probes match the shapes it was explicitly drilled on, so that number measures training coverage, not capability. Off-distribution, it falls apart.
1. It sets problems up correctly and then fumbles the arithmetic.
markdown
17 × 23 → <think> 17 x 20 = 340, 17 x 3 = 51, 340 + 51 = 411 </think> (391)clock → <think> 1060 - 855 = 155 </think> (205)
The decomposition is right every time. The digits are wrong. This is not fixable with
more training data, and the reason is the tokenizer: AuroraGPT's 32k BPE splits numbers
into inconsistent multi-digit chunks — 340 → ['3','40'], 391 → ['39','1'],
51 → ['51']. There is no column alignment for the model to learn. Tokenizers that
split numbers into single digits make carrying learnable; this one does not, and it is
baked into 22.4B tokens of pretraining.
2. On unfamiliar word problems it misreads the question, inventing operations
("16 eggs/day * 3 eggs/day = 48 eggs").
3. A known defect: shallow chat think-blocks leak into reasoning. Forcing 100% think
meant synthesising <think> blocks for ~10k chat turns as a restate-and-plan stub. Those
stubs are ~18% of the corpus and sometimes replace real reasoning on a math prompt:
markdown
<think>The user asks: A robe takes 2 bolts of blue fiber and half that much white fiber. HowI'll answer directly and keep it concrete, starting from: The robe takes 2 bolts...</think>
That is a data-design bug, not a scale limit, and it is the first thing to fix in a v4.
<think> block. There is no gate to
misfire — the previous version trained a 73/27 think/direct split and learned an
inverted gate, skipping reasoning on exactly the hardest prompts.Full-parameter SFT, 2 epochs, sequence-packed, on Kaggle TPU v5e-8 (JAX/Flax + optax, GSPMD sharding): 806 steps, 26.4M tokens, 8.8 minutes, loss 0.6619 → 0.2716.
Corpus: 54,238 conversations, 100% with a think block —
| section | share |
|---|---|
| math (MetaMathQA, GSM8K, Orca-Math, verified-short OpenR1) | 44% |
| chat (SmolTalk + trivial turns) | 19% |
| code (CodeAlpaca task-anchored, MBPP) | 13% |
| targeted drills (distributive multiply, clock, syllogism, algebra traps, multi-step) | 25% |
All 13,326 procedurally generated answers were independently re-validated in Python (0 mismatches) after an earlier build shipped 96 wrong answers from integer division.
markdown
<|system|>{system}<|end|><|user|>{user}<|end|><|assistant|>{reply}<|end|>
EOS is <|end|> (id 5). Context length 2048.
python
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("SmallAICreator/AuroraGPT-Think-v3")model = AutoModelForCausalLM.from_pretrained("SmallAICreator/AuroraGPT-Think-v3")ids = tok.apply_chat_template([{"role": "user", "content": "What is 25% of 120?"}],tokenize=True, add_generation_prompt=True,return_tensors="pt")print(tok.decode(model.generate(ids, max_new_tokens=200)[0][ids.shape[1]:],skip_special_tokens=True))
AuroraGPT-Think-v3.Q8_0.gguf (753MB) is included and runs in llama.cpp / any GGUF
chat app, with the chat template embedded.
Don't ask it to compute. The base model has working tool-calling (15/15 valid tool calls across varied system-prompt wordings) and its chat template declares tools. The correct architecture for this model is reason in the think block, delegate the digits to a calculator tool — which sidesteps the tokenizer problem entirely instead of fighting it.
Made by UltraLabs.