MaliosDark

Nexus-Erebus-50M

README

License: apache-2.0

Model details

Table

Parameters	~54.1M
Architecture	Llama-style decoder
Hidden size	512
Layers	9
Attention heads	8
Vocab size	32,000 (custom digit-aware tokenizer)
Context length	1,024
Precision	bfloat16

The tokenizer keeps digits atomic rather than merging them into BPE units, which preserves the positional structure that integer arithmetic depends on.

Benchmarks

Measured with lm-eval-harness, 0-shot, acc_norm, on the full test sets (no subsampling). ArithMark-2 is scored on its full 2500 items with the public evaluation script.

Table with columns: Task, Items, Nexus-Erebus-50M
Task	Items	Nexus-Erebus-50M
ARC-easy	2,376	44.40
ARC-challenge	1,172	22.70
HellaSwag	10,042	27.05
PIQA	1,838	58.27
ArithMark-2	2,500	52.48
Average

Against the sub-100M field

Published Open SLM Leaderboard values, same five tasks, same full-set protocol.

Table with columns: Model, Params, Average
Model	Params	Average
Nexus-Erebus-50M	54M	40.98
Atom 2.7M	3M	40.43
Supra-1.5-50M-base-exp	52M	39.00
Isabel-50M	54M	38.74
Supra-50M-Base	52M	38.45
Archaea-74M-V1.1	74M

It leads the sub-100M class on ARC-easy and PIQA, and its ArithMark-2 score of 52.48 is the second highest in that class.

Training

Trained from scratch with a custom digit-aware 32k tokenizer, then refined on a curated mix of educational, science, commonsense and reading-comprehension data, plus a large synthetic integer arithmetic set covering addition, subtraction, multiplication, exact division, mixed multi-operator expressions and parenthesised expressions. No benchmark test items were used at any stage.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("MaliosDark/Nexus-Erebus-50M")
model = AutoModelForCausalLM.from_pretrained("MaliosDark/Nexus-Erebus-50M")

prompt = "16 + 4 * 3 ="
print(tok.decode(model.generate(**tok(prompt, return_tensors="pt"), max_new_tokens=6)[0]))

Example outputs

Real, unedited outputs from this checkpoint.

Table with columns: Prompt, Output
Prompt	Output
`Question: What force pulls objects toward the Earth?`	gravity.
`Question: What gas do humans need to breathe to survive?`	oxygen.
`Question: What do we call the process by which plants make food?`	photosynthesis.

License

Apache-2.0.

Built by Ideoa Labs.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

MaliosDark

Model Tree

Base

this model

Input Modalities

Text

Output Modalities