odychlapanis

tosodouliko-gpt

Samples

Prompt «Μια φορά κι έναν καιρό» (in bold), continued by the model with temperature 0.8, top-k 50:

Μια φορά κι έναν καιρό, που για πρώτη φορά κατάφεραν να κάνουν μια ζωή χωρίς να ανησυχεί, θα τον έφερναν σε ένα περιβάλλον με αποτέλεσμα να είναι το κλειδί για την επιτυχία τους.

Μια φορά κι έναν καιρό, ένα μυστήριο, το οποίο έβαζε από την Ελλάδα έναν αιώνα παράδοσης της γης. Οι άνθρωποι που έμοιαζε με τους άλλους και τα δόγματά τους είναι άξιοι του κόσμου που αποφοίτησαν από τα σπίτια της περιοχής.

Μια φορά κι έναν καιρό, η πιο σημαντική, η παγκόσμια κρίση μας δείχνει μια άγνωστη στιγμή. Δεν είναι όμως τίποτα που να μην μας περιμένει, και είμαστε πιο ευάλωτοι.

Use

The weights load as a standard GPT2LMHeadModel. The tokenizer is our own (character-level Greek BPE) and ships as tokenizer.py (standard library only) with tokenizer.json.

python

import importlib.util, torchfrom huggingface_hub import snapshot_downloadfrom transformers import GPT2LMHeadModel path = snapshot_download("odychlapanis/tosodouliko-gpt")spec = importlib.util.spec_from_file_location("tokenizer", f"{path}/tokenizer.py")tokenizer = importlib.util.module_from_spec(spec); spec.loader.exec_module(tokenizer)bpe = tokenizer.GreekBPE.load(f"{path}/tokenizer.json")model = GPT2LMHeadModel.from_pretrained(path).eval() ids = torch.tensor([[bpe.document_id] + bpe.encode("Μια φορά κι έναν καιρό")])   # start with the document tokenout = model.generate(ids, max_new_tokens=60, do_sample=True, temperature=0.8, top_k=50, pad_token_id=0)print(bpe.decode(out[0, 1:].tolist()))

Notes: begin every input with bpe.document_id (it means "a new document starts"), as in training; a sentence break is its own token (bpe.newline_id); the context is 256 tokens.

Model

Parameters: 10.3M (2.6M of them the embedding table)
Architecture: GPT-2 / nanoGPT shape: 6 layers, width 320, 5 heads, GELU MLP (4×), pre-norm LayerNorm, learned positions, tied input/output embeddings, no biases in linear layers
Context: 256 tokens
Tokenizer: 8,000-token character-level BPE trained on our Greek text; word-start marker «▁»; digits split one by one
Alphabet: closed. The 24 Greek letters in both cases, accented vowels and diaereses, digits and « .,;!:'-«» ». Anything else (Latin letters, symbols, emoji) has no token and is dropped by the tokenizer. The model therefore cannot produce a non-Greek character

Training

Data: FineWeb-2 Greek (ell_Grek, ODC-BY), the first file, kept at sentence level and filtered to sentences written entirely in the alphabet above.
Run: 7,854 steps × 64 sequences × 256 tokens ≈ 129M tokens, AdamW (β 0.9/0.95, weight decay 0.1), peak learning rate 1.5e-3 with warmup and cosine decay, no dropout.

Evaluation

All scores are computed from the model's probabilities (no generation), zero-shot.

Greek school benchmark (unreleased). Every item is taken from a Primary (Δημοτικό) or Junior high (Γυμνάσιο) school book:

Grammar: a minimal pair between the book's sentence and a version with one grammatical error. Chance 50%.
Knowledge: a true statement from a History, Geography, Science or Environment book, among versions that differ in one fact. Chance ≈ 28%.
Test split: Primary 361 grammar + 518 knowledge items; Junior 169 + 498.
School score: each task rescaled so 0 = chance and 100 = perfect, then averaged (as nanochat's CORE).

All benchmarks are the Greek versions.

Table with columns: Model, Params, Chars / token ↑, MultiBLiMP, ARC-Easy, HellaSwag, FLORES bpb ↓, School, Primary G / K, Junior G / K
Model	Params	Chars / token ↑	MultiBLiMP	ARC-Easy	HellaSwag	FLORES bpb ↓	School	Primary G / K	Junior G / K
Gemma 3 270M (base)	268M	2.68	92.2	29.9	28.9	0.802	33.2	83.2 / 28.2	84.5 / 26.1

Chars / token: characters per token on the 798 FLORES Greek sentences used for FLORES bpb, each tokenized on its own (higher is better: the same Greek costs fewer tokens).
Public benchmarks: MultiBLiMP Greek (1,096 grammatical / ungrammatical pairs); ARC-Easy Greek (ILSP translation, 2,376 questions) and HellaSwag Greek (ILSP translation, 10,024 contexts), both in cloze form, accuracy with each answer's loss divided by its length in characters (acc_norm); FLORES bpb: bits per byte on the 798 FLORES-200 Greek devtest sentences written fully in the closed alphabet (lower is better; comparable across tokenizers).
School: the school score, centered (0 = chance, 100 = perfect); G / K are grammar and knowledge accuracies.
n/a: not yet measured.

At 10M parameters the model has learned Greek grammar well (MultiBLiMP, school grammar) and almost no knowledge: ARC and HellaSwag are at chance.

Limitations

A small base model: it writes fluent Greek with drifting meaning and will state false things confidently. Do not use it for facts or advice.
Trained on web text, with the biases of web text; no safety tuning.
Greek only, closed alphabet: Latin letters, most symbols and emoji are silently dropped by the tokenizer.
256-token context.

Family

tosodoulikoGPT (10M): this model.
tosodouliGPT (110M): odychlapanis/tosodouli-gpt.

Code (training, tokenizer, evaluation) will be published on GitHub.

Citation

bibtex

@misc{chlapanis2026tosodoulikogpt,  author       = {Chlapanis, Odysseas S.},  title        = {tosodoulikoGPT: a 10M-parameter Greek language model trained from scratch},  year         = {2026},  publisher    = {Hugging Face},  howpublished = {\url{https://huggingface.co/odychlapanis/tosodouliko-gpt}}}

Acknowledgements

Training data: FineWeb-2 (Hugging Face, ODC-BY).
Architecture template: nanoGPT (Andrej Karpathy).
Teaching approach: Build a Large Language Model (From Scratch) (Sebastian Raschka).
Evaluation: MultiBLiMP (Jumelet et al.); Greek ARC and HellaSwag translations and FLORES-200 Greek (ILSP / Athena Research Center); FLORES-200 (NLLB Team, Meta AI).

Licence

MIT. The training data (FineWeb-2) is ODC-BY and is not redistributed.

Author: Odysseas S. Chlapanis.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

odychlapanis

Model Tree

Base

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Samples

Prompt «Μια φορά κι έναν καιρό» (in bold), continued by the model with temperature 0.8, top-k 50:

Μια φορά κι έναν καιρό, που για πρώτη φορά κατάφεραν να κάνουν μια ζωή χωρίς να ανησυχεί, θα τον έφερναν σε ένα περιβάλλον με αποτέλεσμα να είναι το κλειδί για την επιτυχία τους.

Μια φορά κι έναν καιρό, ένα μυστήριο, το οποίο έβαζε από την Ελλάδα έναν αιώνα παράδοσης της γης. Οι άνθρωποι που έμοιαζε με τους άλλους και τα δόγματά τους είναι άξιοι του κόσμου που αποφοίτησαν από τα σπίτια της περιοχής.

Μια φορά κι έναν καιρό, η πιο σημαντική, η παγκόσμια κρίση μας δείχνει μια άγνωστη στιγμή. Δεν είναι όμως τίποτα που να μην μας περιμένει, και είμαστε πιο ευάλωτοι.

Use

The weights load as a standard GPT2LMHeadModel. The tokenizer is our own (character-level Greek BPE) and ships as tokenizer.py (standard library only) with tokenizer.json.

python

import importlib.util, torchfrom huggingface_hub import snapshot_downloadfrom transformers import GPT2LMHeadModel path = snapshot_download("odychlapanis/tosodouliko-gpt")spec = importlib.util.spec_from_file_location("tokenizer", f"{path}/tokenizer.py")tokenizer = importlib.util.module_from_spec(spec); spec.loader.exec_module(tokenizer)bpe = tokenizer.GreekBPE.load(f"{path}/tokenizer.json")model = GPT2LMHeadModel.from_pretrained(path).eval() ids = torch.tensor([[bpe.document_id] + bpe.encode("Μια φορά κι έναν καιρό")])   # start with the document tokenout = model.generate(ids, max_new_tokens=60, do_sample=True, temperature=0.8, top_k=50, pad_token_id=0)print(bpe.decode(out[0, 1:].tolist()))

Notes: begin every input with bpe.document_id (it means "a new document starts"), as in training; a sentence break is its own token (bpe.newline_id); the context is 256 tokens.

Model

Parameters: 10.3M (2.6M of them the embedding table)
Architecture: GPT-2 / nanoGPT shape: 6 layers, width 320, 5 heads, GELU MLP (4×), pre-norm LayerNorm, learned positions, tied input/output embeddings, no biases in linear layers
Context: 256 tokens
Tokenizer: 8,000-token character-level BPE trained on our Greek text; word-start marker «▁»; digits split one by one
Alphabet: closed. The 24 Greek letters in both cases, accented vowels and diaereses, digits and « .,;!:'-«» ». Anything else (Latin letters, symbols, emoji) has no token and is dropped by the tokenizer. The model therefore cannot produce a non-Greek character

Training

Data: FineWeb-2 Greek (ell_Grek, ODC-BY), the first file, kept at sentence level and filtered to sentences written entirely in the alphabet above.
Run: 7,854 steps × 64 sequences × 256 tokens ≈ 129M tokens, AdamW (β 0.9/0.95, weight decay 0.1), peak learning rate 1.5e-3 with warmup and cosine decay, no dropout.

Evaluation

All scores are computed from the model's probabilities (no generation), zero-shot.

Greek school benchmark (unreleased). Every item is taken from a Primary (Δημοτικό) or Junior high (Γυμνάσιο) school book:

Grammar: a minimal pair between the book's sentence and a version with one grammatical error. Chance 50%.
Knowledge: a true statement from a History, Geography, Science or Environment book, among versions that differ in one fact. Chance ≈ 28%.
Test split: Primary 361 grammar + 518 knowledge items; Junior 169 + 498.
School score: each task rescaled so 0 = chance and 100 = perfect, then averaged (as nanochat's CORE).

All benchmarks are the Greek versions.

Table with columns: Model, Params, Chars / token ↑, MultiBLiMP, ARC-Easy, HellaSwag, FLORES bpb ↓, School, Primary G / K, Junior G / K
Model	Params	Chars / token ↑	MultiBLiMP	ARC-Easy	HellaSwag	FLORES bpb ↓	School	Primary G / K	Junior G / K
Gemma 3 270M (base)	268M	2.68	92.2	29.9	28.9	0.802	33.2	83.2 / 28.2	84.5 / 26.1

Chars / token: characters per token on the 798 FLORES Greek sentences used for FLORES bpb, each tokenized on its own (higher is better: the same Greek costs fewer tokens).
Public benchmarks: MultiBLiMP Greek (1,096 grammatical / ungrammatical pairs); ARC-Easy Greek (ILSP translation, 2,376 questions) and HellaSwag Greek (ILSP translation, 10,024 contexts), both in cloze form, accuracy with each answer's loss divided by its length in characters (acc_norm); FLORES bpb: bits per byte on the 798 FLORES-200 Greek devtest sentences written fully in the closed alphabet (lower is better; comparable across tokenizers).
School: the school score, centered (0 = chance, 100 = perfect); G / K are grammar and knowledge accuracies.
n/a: not yet measured.

At 10M parameters the model has learned Greek grammar well (MultiBLiMP, school grammar) and almost no knowledge: ARC and HellaSwag are at chance.

Limitations

A small base model: it writes fluent Greek with drifting meaning and will state false things confidently. Do not use it for facts or advice.
Trained on web text, with the biases of web text; no safety tuning.
Greek only, closed alphabet: Latin letters, most symbols and emoji are silently dropped by the tokenizer.
256-token context.

Family

tosodoulikoGPT (10M): this model.
tosodouliGPT (110M): odychlapanis/tosodouli-gpt.

Code (training, tokenizer, evaluation) will be published on GitHub.

Citation

bibtex

@misc{chlapanis2026tosodoulikogpt,  author       = {Chlapanis, Odysseas S.},  title        = {tosodoulikoGPT: a 10M-parameter Greek language model trained from scratch},  year         = {2026},  publisher    = {Hugging Face},  howpublished = {\url{https://huggingface.co/odychlapanis/tosodouliko-gpt}}}

Acknowledgements

Training data: FineWeb-2 (Hugging Face, ODC-BY).
Architecture template: nanoGPT (Andrej Karpathy).
Teaching approach: Build a Large Language Model (From Scratch) (Sebastian Raschka).
Evaluation: MultiBLiMP (Jumelet et al.); Greek ARC and HellaSwag translations and FLORES-200 Greek (ILSP / Athena Research Center); FLORES-200 (NLLB Team, Meta AI).

Licence

MIT. The training data (FineWeb-2) is ODC-BY and is not redistributed.

Author: Odysseas S. Chlapanis.

tosodouliko-gpt

README

Samples

Use

Model

Training

Evaluation

Limitations

Family

Citation

Acknowledgements

Licence

Explore FriendliAI today

README

Samples

Use

Model

Training

Evaluation

Limitations

Family

Citation

Acknowledgements

Licence