GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
Thox-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Thox-ai
Model Tree
Input Modalities
Output Modalities
Supported Functionality
This is a base / completion model. It is not instruction-tuned, has no chat template, carries no THOX persona or curated facts, and will not reliably answer general-knowledge questions. It is a foundation for SFT/LoRA work and a local building block at the edge. Prompt it in completion style and use a repetition penalty of ~1.2+ to avoid degeneration.
| Field | Value |
|---|---|
| Parameters | 123,587,328 |
| Architecture | Llama-family decoder (RMSNorm, SwiGLU, RoPE) |
| Layers / hidden / heads | 12 / 768 / 12 (full MHA, 12 KV heads) |
| Head dim / FFN | 64 / 2048 |
| Context length | 1024 |
| Tokenizer | GPT-2 / r50k_base BPE, vocab 50,304 (padded) |
| Tied embeddings | yes |
| RoPE theta | 10,000 |
| Precision | fp16 weights (BF16 mixed-precision training) |
| Training steps | 100,000 |
| Best validation loss | 2.6099 |
| Final validation loss | 2.6459 |
| Hardware | 1 × NVIDIA RTX 4060 Ti 16 GB |
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizermodel = AutoModelForCausalLM.from_pretrained("Thox-ai/thox-micro-125m", dtype=torch.float32)tok = AutoTokenizer.from_pretrained("Thox-ai/thox-micro-125m")ids = tok("The capital of France is", return_tensors="pt")out = model.generate(**ids, max_new_tokens=32, do_sample=True,temperature=0.8, top_k=40, repetition_penalty=1.3)print(tok.decode(out[0], skip_special_tokens=True))
For device / CPU inference use the GGUF builds:
bash
llama-completion -m thox-micro-125m.q4_0.gguf \-p "The capital of France is" -n 32 --repeat-penalty 1.3 --temp 0.8
Published in Thox-ai/thox-micro-125m-GGUF:
| Quant | Size | Note |
|---|---|---|
| Q4_0 | 77.6 MB | Device target — scalar-safe on ARMv6 (no NEON needed) |
| Q4_K_M | 80.7 MB | Higher-quality alternate (ARMv7+/x86) |
| Q8_0 | 127.0 MB | Higher quality, ~1.6× the weights |
| F16 | 237.5 MB | Re-quant source |
Q4_0 perplexity 22.60 vs F16 21.41 (~6%) on a neutral English sample — clean quantization. Unlike THOX's 327M (whose FFN dim of 3413 forces K-quants only), this model's FFN dim of 2048 divides evenly, so the ARM-friendly legacy Q4_0 and Q8_0 formats both apply.
Note: some files named
thox-micro-125m.*.ggufwith uppercase quant suffixes also exist in this repository from an earlier upload. The canonical GGUFs are the lowercase-named files inThox-ai/thox-micro-125m-GGUF— that is what the THOX device images pin by digest.
Trained from scratch by THOX.ai LLC — no base model, no distillation. THOX IP-017.
Inventors of record are listed in NOTICE.