Quick start
ollama create tavik-1.3 -f Modelfileollama run tavik-1.3 "What is 23 * 17?"
Use the Modelfile from this repository (chat template, repeat_penalty 1.0: a repeat penalty makes the
step-by-step math worse, because the steps repeat numbers on purpose).
What changed from Tavik-1
- Digits are single tokens. All tokenizer merges that combined digits were removed, so
4728 is always
4 7 2 8. This makes arithmetic learnable for a small model.
- Fine-tuning mix (365,000 examples, about 100M tokens, one pass):
Table with columns: Part, Examples| Part | Examples |
|---|
| Step-by-step arithmetic up to 4 digits (+, -, *, /, brackets, percent, word problems) | 120,000 |
| Step-by-step reasoning (sequences, ordering, days of the week, letter counting, units, ages, decimals) | 50,000 |
| Original Tavik-1 chats, so it keeps its chat ability | 180,000 |
| Identity and Convergent family | 15,000 |
- Training: 3,200 steps, learning rate 2e-4 with cosine decay, context 1024, on one RTX 3060 Ti in about
25 minutes (MakeAI). Validation loss on held-out data of the same mix: 1.470, token accuracy 66.6 %.
Evaluation (new problems, none of them in the training data)
Table with columns: Test, Tavik-1, Tavik-1.3| Test | Tavik-1 | Tavik-1.3 |
|---|
| Arithmetic, 420 problems (answer must match exactly) | 0 % | 25.2 % |
| - multiplication | 0 % | 55.7 % |
| - percent | 0 % | 55.6 % |
| - word problems | 0 % | 27.9 % |
| - addition | 0 % | 23.0 % |
| - order of operations | 0 % | 11.5 % |
| - subtraction | 0 % | 3.3 % |
| - division | 0 % | 3.3 % |
| Reasoning, 100 problems | 0 | 52 |
| 5-digit addition, 3x3-digit multiplication (outside the training range) | 0 % | 0 % |
| Python tasks / valid HTML | 0 / 0 | 0 / 0 |
Arithmetic and reasoning only count when the final Answer: line is exactly right. Tavik-1 never wrote
that line, so its 0 % is partly a format effect.
Example:
User: What is 23 * 17?23 * 17 = 23 * 10 + 23 * 723 * 10 = 23 * 1 * 10...230 + 161 = 391Answer: 391
Limitations
- Facts are often wrong (it called the largest planet "the Solar Cycle").
- Longer creative texts such as poems fall into repetition loops. A higher repeat penalty (1.1) did not fix
this in our test and lowered the arithmetic score to 21.7 %.
- Subtraction and division are still weak; small mistakes in one step make the final answer wrong.
- No code, English only.
License
Apache 2.0.