murai-labs

agaram-2.5b

README

License: apache-2.0

At a glance

Table

Parameters	2,525,757,696 (28 layers, hidden 2,304, 36 heads of 64, SwiGLU 9,216, tied embeddings, no biases)
Tokenizer	byte-level BPE, 64,000 tokens, no normaliser (keeps line breaks); 1.55 tokens per Tamil word
Context	4,096 tokens in the stable stage, 8,192 in the decay stage (RoPE base 500,000)
Training	50.0B tokens: 82,970 stable steps + 12,398 decay steps at 524,288 tokens per step, warmup-stable-decay, peak learning rate 3e-4
Data	Tamil 58.2%, English 19.8%, math 10%, code 7%, parallel 2%, instruction data 3% (decay stage only)
Hardware	8x NVIDIA H100 80GB (Lambda), 2026-10-02 to 2026-10-08, one run, no restarts
Final validation loss	1.9025 (decay-stage held-out set)
Licence	Apache-2.0 (upstream data terms under "Licence")

Attention never crosses a document boundary (packed document masking), and every document ends with an end-of-text token (id 0).

Evaluation

lm-eval on our hardware, same tasks and settings for every model; Sarvam-1 is a 2B Indic base model.

Table with columns: Task, Agaram 2.5B base, Sarvam-1
Task	Agaram 2.5B base	Sarvam-1
FLORES en→ta / ta→en, chrF (5-shot)	55.1 / 55.5	49.2 / 44.2
IN22-Conv en→ta / ta→en, chrF	40.8 / 46.0	37.4 / 37.9
IN22-Gen en→ta / ta→en, chrF	48.8 / 54.3	45.9 / 42.7
MILU Tamil (5-shot)	29.6	28.8
Belebele (ta)	27.8	58.3
INCLUDE (Tamil)	26.9	35.5
MMLU (ta)	25.5	32.6
ARC (ta)	21.9	24.4
HellaSwag (ta)	26.0	26.2
XCOPA (ta)	62.0	62.0
MultiBLiMP (ta)	99.0	97.9

It translates better than Sarvam-1 in all six directions and is at or near chance on reading comprehension and knowledge. Some FLORES and IN22-Gen English test sentences occur in the pretraining text; scores on the clean subsets are within 1.1 chrF.

Data disclosures

Public datasets on Hugging Face with open licences, Tamil books OCR'd for this project, and NVIDIA datasets used under their agreement (training only; nothing derived from them is redistributed). Near-duplicate removal took the corpus from 87.7M to 61.3M documents, and 42,001 documents overlapping 20 benchmark and evaluation sets were removed.
The decay stage's instruction data includes two public sets produced by frontier models before we used them: a Tamil translation of Alpaca (51,699 rows; answers generated by OpenAI's text-davinci-003; card licence GPL-3.0, upstream Stanford Alpaca data CC BY-NC 4.0) and Tamil Orca (93,932 rows; GPT-4 and Gemini answers; Apache-2.0).
OCR text from bodhan-ai/indic-ocr (0.17% of stable and 0.56% of decay tokens; Indic Open Model License) and from Sarvam AI's OCR API (0.10% and 0.33%; Sarvam terms of service, section 10.5(a)). We have asked both companies in writing and disclose the use here.

Licence

The model weights, tokenizer and configuration are released under the Apache License, Version 2.0 (see LICENSE), so that anyone can use them, including for commercial work. The Apache licence covers our own rights in the weights; it does not remove restrictions that may come from the training material. If you redistribute these files or a modified version, keep the NOTICE file with them, as section 4(d) of the licence requires. The NOTICE file names the authors and the material below.

Some of the pretraining data has its own terms (see "Data disclosures"):

Tamil Alpaca (51,699 rows in the decay stage). The Hugging Face card says GPL-3.0. The upstream Stanford Alpaca data is CC BY-NC 4.0, and Stanford asks that models trained on it be used for research only.
Tamil Orca (93,932 rows in the decay stage). Apache-2.0; its answers were generated by GPT-4 and Gemini.
OCR text from bodhan-ai/indic-ocr (Indic Open Model License) and from Sarvam AI's OCR API (Sarvam terms of service, section 10.5(a)). Together these are about 0.27% of stable-stage and 0.89% of decay-stage tokens. We asked both providers for permission in writing on 10 October 2026; neither has granted it.

This is not legal advice.

Creators

Murai Labs and Ramchand Kumaresan.

Citation

If you use Agaram 2.5B or build on it, please credit Murai Labs and Ramchand Kumaresan and cite the technical report: https://arxiv.org/abs/2610.18420

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

murai-labs

Model Tree

Base

this model

Input Modalities

Text

Output Modalities