Model details
- Base model: HuggingFaceTB/SmolLM2-135M
- Parameters: 135M
- Context length: 8192
- Training: 6 epochs on 182k examples, about 1.1B tokens
- Mix: 61 percent chain of thought data including Bespoke Stratos, OpenThoughts, NuminaMath, Orca Math, OpenR1 Math, MathInstruct, plus general chat data from SmolTalk, Cosmopedia, FineTome and UltraChat
- Identity: trained to identify as Adurite1 from Maximalist Labs
- Hardware: TPU v6e-1
Training
Trained on TPU with max length 1024, per device batch 32, AdaFactor, cosine schedule, warmup 500 steps, weight decay 0.01.
Final loss 1.337 after 33732 steps.
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("MLVXN/Adurite1-135M-COT")
model = AutoModelForCausalLM.from_pretrained("MLVXN/Adurite1-135M-COT")
prompt = "<|im_start|>user\nWhat is 12*8? Think step by step.<|im_end|>\n<|im_start|>assistant to=self thinking\n"
inputs = tok(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True))
Or use the GGUF file with llama.cpp:
llama-cli -m adurite1-135m-cot-f16.gguf -p "What is 12*8? Think step by step."
GGUF
GGUF weights are available in this repo:
adurite1-135m-cot-f16.gguf 313 MB full precision
adurite1-135m-cot-q4_k_m.gguf quantized 4 bit (when available)
Evaluation
Work in progress. Early MMLU and GSM8K runs are being added. MMLU is expected around 33 to 36 percent, GSM8K around 45 to 50 percent with chain of thought prompting.
Limitations
This is a 135M model. It can do short reasoning traces of 2 to 4 steps but will fail on hard math and long logic problems. It may repeat or miss facts. Use with care and check outputs for important tasks.
License
Apache 2.0
Citation
@misc{adurite1-135m-cot,
title = {Adurite1-135M-COT},
author = {MLVXN},
year = {2026},
url = {https://huggingface.co/MLVXN/Adurite1-135M-COT}
}