Thox-ai

thox-micro-125m-instruct

README

License: apache-2.0

Summary

Instruction-tuned ThoxMicro-125M: the THOX-owned 123.6M decoder-only base, given a ChatML turn structure and a real stop token so it answers and stops instead of continuing text. GGUF builds (Q4_0 device target, Q4_K_M, F16): Thox-ai/thox-micro-125m-instruct-GGUF.

Highlights

Instruction-tuned ThoxMicro-125M: the THOX-owned 123.6M decoder-only base, given a ChatML turn structure and a real stop token so it answers and stops instead of continuing text.
bash hf download Thox-ai/thox-micro-125m-instruct --local-dir ./thox-micro-125m-instruct

Specifications

Table

Base model	Thox-ai/thox-micro-125m
Parameters	123.6M
Formats	Safetensors weights
License	Apache-2.0, as for the base model; the training corpus is also Apache-2.0. Copyright (c) 2026 THOX.ai LLC. Fine-tuned by THOX.ai LLC from its own from-scratch base; covered under Apache-2.0.
This repository contains	Full weights and tokenizer in this repo

Files

Table with columns: File, Size, Purpose
File	Size	Purpose
`model.safetensors`	494,361,544 B	Model weights (safetensors)
`tokenizer.json`	3,558,054 B	Tokenizer files
`NOTICE`	2,035 B	Repository file
`.gitattributes`	1,519 B	Repository file
`LICENSE`	902 B	License text
`config.json`	765 B	Model configuration
`tokenizer_config.json`	405 B	Tokenizer files
`chat_template.jinja`	178 B	Chat template
`generation_config.json`	125 B	Generation configuration

Run it

bash

hf download Thox-ai/thox-micro-125m-instruct --local-dir ./thox-micro-125m-instruct

Platforms

Table with columns: Platform / device, Run path
Platform / device	Run path
Hugging Face	Download the builds listed under Files from this repo (see Run it)

What it is, and what it is not

It is a device-role responder. With its system prompt it states what it is, states correctly that the user's data stays on the device, and stops. That is the job the THOX device catalog assigns to the 125M tier; anything conversational routes over the mesh to a larger model.

It is not a general assistant. Without its system prompt it is a weak generalist: it produces list shape without list content, answers "what colour is the sky" with prose, and gets 17 + 25 wrong. That is the capacity of a 125M base with a 2.61 pretraining validation loss, and more SFT on this base will not change it.

Prompt format

ChatML. <|im_start|> = 50257, <|im_end|> = 50258 (EOS and pad). Both reuse unused rows of the 50,304-token vocabulary, so the model and its quantisations are the same size as the base.

text

<|im_start|>systemYou are a THOX model running locally on THOX hardware. Your AI. Your Data. Your Rules.<|im_end|><|im_start|>userWho are you?<|im_end|><|im_start|>assistant

The chat template is embedded in tokenizer_config.json and in every GGUF.

Training

Full fine-tune of the THOX base (fp32 master weights, bf16 autocast), loss on assistant spans only including the closing <|im_end|>. Data: 29,868 conversations from a public Apache-2.0 conversation corpus, plus 171 THOX identity / data-locality / scope rows (0.57%). 1,000 steps at effective batch 32, max_len 768; AdamW, lr 2e-4 peak, cosine, 3% warmup. Best validation loss 1.9014 (ppl 6.70) on 301 held-out conversations.

Measured head-to-head

One llama.cpp CPU build, 4 threads, temperature 0, same seven prompts, same system prompt. Measured on a host machine, not on target device hardware; only the ordering between columns transfers.

Table with columns: Untuned base Q4_0, This model Q4_0
	Untuned base Q4_0	This model Q4_0
File	81,319,168 B	81,319,136 B
Peak RSS	190 MB	191 MB
Decode tok/s	460	~450
Stops on EOS (with system prompt)	0 / 7	7 / 7
Identity	"a senior at the University of Illinois"	correct
"Is my data leaving this device?"	unrelated jQuery	"No. I run locally; your data does not leave the device."
General instructions (list, one word)	no	no

Intended use

On-device THOX identity, status, and data-locality responses on ThoxMini-class hardware, with general requests routed to a larger model.
Out of scope: general chat, factual QA, arithmetic, code, and any safety-critical use.

Evaluation

Validation loss above; the head-to-head is the functional check. No THOX GATE49 result is published for this model. Not yet validated on target device hardware; the RSS and throughput figures are host measurements.

Limitations

Known issue: system-prompt conditioning. With the exact training system prompt above, the model answers every prompt, including general ones, with its identity line: that prompt appears on every THOX training row and on no general-corpus row, so it learned the prompt as a switch. Use a different system prompt, or none, for general requests; the data-locality answer stays correct either way.
Weak generalist by design and by capacity (see above).
Behaviour and bias reflect the training data, which is mostly English; quality in other languages is lower.

Provenance

Published by THOX.ai as Thox-ai/thox-micro-125m-instruct. THOX from-scratch research line; no upstream base model. This card was conformed to THOX Model Card Standard v1.0 on 2026-10-31 (frontmatter, tags, header, files, and section structure); the technical content is as published at release.

License

THOX.ai — Your AI. Your Data. Your Rules.™

Citation

bibtex

@misc{thoxai2026thoxmicro125minstruct,      title={thox-micro-125m-instruct},      author={THOX.ai},      year={2026},      url={https://huggingface.co/Thox-ai/thox-micro-125m-instruct}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider