odychlapanis

tosodouli-gpt

Samples

Prompt «Μια φορά κι έναν καιρό» (in bold), continued by the model with temperature 0.8, top-k 50:

Μια φορά κι έναν καιρό ήταν ένα κορίτσι που το έλεγαν Ευδοκία. Η Ευδοκία ήταν ένα κορίτσι με το οποίο η ζωή δεν ήταν εύκολη, αλλά ευτυχώς που ήταν γιατί δεν ήταν όμορφη. Όταν το κορίτσι πήγε να κοιμηθεί της έλεγε πως δεν θα έβγαινε με κανέναν γιατί είχε βρει δουλειά. Η Ευδοκία την αγαπούσε πολύ και της έκανε όλα τα χατίρια.

Μια φορά κι έναν καιρό ήταν ένας άνθρωπος που ήξερε να λέει μόνο μια λέξη. Τον έλεγαν Οδυσσέα και ήταν ένας μεγάλος ψαράς. Όταν τον αντίκριζε ο ήλιος ήταν ολοκάθαρος. -Δε θα πας πουθενά, του απάντησε κι έφυγε τρέχοντας.

Μια φορά κι έναν καιρό ήταν μια πανέμορφη πριγκίπισσα. Την έλεγαν Σαλώμη και ήταν όμορφη σαν πριγκίπισσα. Είχε όλα τα χαρακτηριστικά της πριγκίπισσας της καρδιάς της και ήταν τόσο πολύ όμορφη πάνω στο όμορφο σώμα της ώστε ακόμα και το όνομα της δεν της χωρούσε.

Use

The weights load as a standard GPT2LMHeadModel. The tokenizer is our own (character-level Greek BPE) and ships as tokenizer.py (standard library only) with tokenizer.json.

python

import importlib.util, torchfrom huggingface_hub import snapshot_downloadfrom transformers import GPT2LMHeadModel path = snapshot_download("odychlapanis/tosodouli-gpt")spec = importlib.util.spec_from_file_location("tokenizer", f"{path}/tokenizer.py")tokenizer = importlib.util.module_from_spec(spec); spec.loader.exec_module(tokenizer)bpe = tokenizer.GreekBPE.load(f"{path}/tokenizer.json")model = GPT2LMHeadModel.from_pretrained(path).eval() ids = torch.tensor([[bpe.document_id] + bpe.encode("Μια φορά κι έναν καιρό")])   # start with the document tokenout = model.generate(ids, max_new_tokens=80, do_sample=True, temperature=0.8, top_k=50, pad_token_id=0)print(bpe.decode(out[0, 1:].tolist()))

Notes: begin every input with bpe.document_id (it means "a new document starts"), as in training; a sentence break is its own token (bpe.newline_id); the context is 1,024 tokens.

Model

Parameters: 110.3M (24.6M of them the embedding table)
Architecture: GPT-2 / nanoGPT shape: 12 layers, width 768, 12 heads, GELU MLP (4×), pre-norm LayerNorm, learned positions, tied input/output embeddings, no biases in linear layers
Context: 1,024 tokens
Tokenizer: 32,000-token character-level BPE trained on our Greek text; word-start marker «▁»; digits split one by one
Alphabet: closed. The 24 Greek letters in both cases, accented vowels and diaereses, digits and « .,;!:'-«» ». Anything else (Latin letters, symbols, emoji) has no token and is dropped by the tokenizer. The model therefore cannot produce a non-Greek character

Training

Data: FineWeb-2 Greek (ell_Grek, ODC-BY), ten files, kept at sentence level and filtered to sentences written entirely in the alphabet above: 9.6B tokens.
Run: 25,000 steps × 480 sequences × 1,024 tokens ≈ 12.3B tokens, AdamW (β 0.9/0.95, weight decay 0.1), peak learning rate 6e-4 with warmup and cosine decay, no dropout.

Evaluation

All scores are computed from the model's probabilities (no generation), zero-shot.

Greek school benchmark (unreleased). Every item is taken from a Primary (Δημοτικό) or Junior high (Γυμνάσιο) school book:

Grammar: a minimal pair between the book's sentence and a version with one grammatical error. Chance 50%.
Knowledge: a true statement from a History, Geography, Science or Environment book, among versions that differ in one fact. Chance ≈ 28%.
Test split: Primary 361 grammar + 518 knowledge items; Junior 169 + 498.
School score: each task rescaled so 0 = chance and 100 = perfect, then averaged (as nanochat's CORE).

All benchmarks are the Greek versions.

Table with columns: Model, Params, Chars / token ↑, MultiBLiMP, ARC-Easy, HellaSwag, FLORES bpb ↓, School, Primary G / K, Junior G / K
Model	Params	Chars / token ↑	MultiBLiMP	ARC-Easy	HellaSwag	FLORES bpb ↓	School	Primary G / K	Junior G / K
Gemma 3 270M (base)	268M	2.68	92.2	29.9	28.9	0.802	33.2	83.2 / 28.2	84.5 / 26.1

Chars / token: characters per token on the 798 FLORES Greek sentences used for FLORES bpb, each tokenized on its own (higher is better: the same Greek costs fewer tokens).
Public benchmarks: MultiBLiMP Greek (1,096 grammatical / ungrammatical pairs); ARC-Easy Greek (ILSP translation, 2,376 questions) and HellaSwag Greek (ILSP translation, 10,024 contexts), both in cloze form, accuracy with each answer's loss divided by its length in characters (acc_norm); FLORES bpb: bits per byte on the 798 FLORES-200 Greek devtest sentences written fully in the closed alphabet (lower is better; comparable across tokenizers).
School: the school score, centered (0 = chance, 100 = perfect); G / K are grammar and knowledge accuracies.
n/a: not yet measured.

At 110M parameters the model has nearly solved Greek grammar (MultiBLiMP, school grammar) and has learned some knowledge (half of the Primary knowledge items), while ARC and HellaSwag are only a few points above chance.

Limitations

A small base model: it writes fluent Greek but loses the thread, and will state false things confidently. Do not use it for facts or advice.
Trained on web text, with the biases of web text; no safety tuning.
Greek only, closed alphabet: Latin letters, most symbols and emoji are silently dropped by the tokenizer.
1,024-token context.

Family

tosodoulikoGPT (10M): odychlapanis/tosodouliko-gpt.
tosodouliGPT (110M): this model.

Code (training, tokenizer, evaluation) will be published on GitHub.

Citation

bibtex

@misc{chlapanis2026tosodouligpt,  author       = {Chlapanis, Odysseas S.},  title        = {tosodouliGPT: a 110M-parameter Greek language model trained from scratch},  year         = {2026},  publisher    = {Hugging Face},  howpublished = {\url{https://huggingface.co/odychlapanis/tosodouli-gpt}}}

Acknowledgements

Training data: FineWeb-2 (Hugging Face, ODC-BY).
Architecture template: nanoGPT (Andrej Karpathy).
Teaching approach: Build a Large Language Model (From Scratch) (Sebastian Raschka).
Evaluation: MultiBLiMP (Jumelet et al.); Greek ARC and HellaSwag translations and FLORES-200 Greek (ILSP / Athena Research Center); FLORES-200 (NLLB Team, Meta AI).

Licence

MIT. The training data (FineWeb-2) is ODC-BY and is not redistributed.

Author: Odysseas S. Chlapanis.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

odychlapanis

Model Tree

Base

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Samples

Prompt «Μια φορά κι έναν καιρό» (in bold), continued by the model with temperature 0.8, top-k 50:

Μια φορά κι έναν καιρό ήταν ένα κορίτσι που το έλεγαν Ευδοκία. Η Ευδοκία ήταν ένα κορίτσι με το οποίο η ζωή δεν ήταν εύκολη, αλλά ευτυχώς που ήταν γιατί δεν ήταν όμορφη. Όταν το κορίτσι πήγε να κοιμηθεί της έλεγε πως δεν θα έβγαινε με κανέναν γιατί είχε βρει δουλειά. Η Ευδοκία την αγαπούσε πολύ και της έκανε όλα τα χατίρια.

Μια φορά κι έναν καιρό ήταν ένας άνθρωπος που ήξερε να λέει μόνο μια λέξη. Τον έλεγαν Οδυσσέα και ήταν ένας μεγάλος ψαράς. Όταν τον αντίκριζε ο ήλιος ήταν ολοκάθαρος. -Δε θα πας πουθενά, του απάντησε κι έφυγε τρέχοντας.

Μια φορά κι έναν καιρό ήταν μια πανέμορφη πριγκίπισσα. Την έλεγαν Σαλώμη και ήταν όμορφη σαν πριγκίπισσα. Είχε όλα τα χαρακτηριστικά της πριγκίπισσας της καρδιάς της και ήταν τόσο πολύ όμορφη πάνω στο όμορφο σώμα της ώστε ακόμα και το όνομα της δεν της χωρούσε.

Use

The weights load as a standard GPT2LMHeadModel. The tokenizer is our own (character-level Greek BPE) and ships as tokenizer.py (standard library only) with tokenizer.json.

python

import importlib.util, torchfrom huggingface_hub import snapshot_downloadfrom transformers import GPT2LMHeadModel path = snapshot_download("odychlapanis/tosodouli-gpt")spec = importlib.util.spec_from_file_location("tokenizer", f"{path}/tokenizer.py")tokenizer = importlib.util.module_from_spec(spec); spec.loader.exec_module(tokenizer)bpe = tokenizer.GreekBPE.load(f"{path}/tokenizer.json")model = GPT2LMHeadModel.from_pretrained(path).eval() ids = torch.tensor([[bpe.document_id] + bpe.encode("Μια φορά κι έναν καιρό")])   # start with the document tokenout = model.generate(ids, max_new_tokens=80, do_sample=True, temperature=0.8, top_k=50, pad_token_id=0)print(bpe.decode(out[0, 1:].tolist()))

Notes: begin every input with bpe.document_id (it means "a new document starts"), as in training; a sentence break is its own token (bpe.newline_id); the context is 1,024 tokens.

Model

Parameters: 110.3M (24.6M of them the embedding table)
Architecture: GPT-2 / nanoGPT shape: 12 layers, width 768, 12 heads, GELU MLP (4×), pre-norm LayerNorm, learned positions, tied input/output embeddings, no biases in linear layers
Context: 1,024 tokens
Tokenizer: 32,000-token character-level BPE trained on our Greek text; word-start marker «▁»; digits split one by one
Alphabet: closed. The 24 Greek letters in both cases, accented vowels and diaereses, digits and « .,;!:'-«» ». Anything else (Latin letters, symbols, emoji) has no token and is dropped by the tokenizer. The model therefore cannot produce a non-Greek character

Training

Data: FineWeb-2 Greek (ell_Grek, ODC-BY), ten files, kept at sentence level and filtered to sentences written entirely in the alphabet above: 9.6B tokens.
Run: 25,000 steps × 480 sequences × 1,024 tokens ≈ 12.3B tokens, AdamW (β 0.9/0.95, weight decay 0.1), peak learning rate 6e-4 with warmup and cosine decay, no dropout.

Evaluation

All scores are computed from the model's probabilities (no generation), zero-shot.

Greek school benchmark (unreleased). Every item is taken from a Primary (Δημοτικό) or Junior high (Γυμνάσιο) school book:

Grammar: a minimal pair between the book's sentence and a version with one grammatical error. Chance 50%.
Knowledge: a true statement from a History, Geography, Science or Environment book, among versions that differ in one fact. Chance ≈ 28%.
Test split: Primary 361 grammar + 518 knowledge items; Junior 169 + 498.
School score: each task rescaled so 0 = chance and 100 = perfect, then averaged (as nanochat's CORE).

All benchmarks are the Greek versions.

Table with columns: Model, Params, Chars / token ↑, MultiBLiMP, ARC-Easy, HellaSwag, FLORES bpb ↓, School, Primary G / K, Junior G / K
Model	Params	Chars / token ↑	MultiBLiMP	ARC-Easy	HellaSwag	FLORES bpb ↓	School	Primary G / K	Junior G / K
Gemma 3 270M (base)	268M	2.68	92.2	29.9	28.9	0.802	33.2	83.2 / 28.2	84.5 / 26.1

Chars / token: characters per token on the 798 FLORES Greek sentences used for FLORES bpb, each tokenized on its own (higher is better: the same Greek costs fewer tokens).
Public benchmarks: MultiBLiMP Greek (1,096 grammatical / ungrammatical pairs); ARC-Easy Greek (ILSP translation, 2,376 questions) and HellaSwag Greek (ILSP translation, 10,024 contexts), both in cloze form, accuracy with each answer's loss divided by its length in characters (acc_norm); FLORES bpb: bits per byte on the 798 FLORES-200 Greek devtest sentences written fully in the closed alphabet (lower is better; comparable across tokenizers).
School: the school score, centered (0 = chance, 100 = perfect); G / K are grammar and knowledge accuracies.
n/a: not yet measured.

Limitations

A small base model: it writes fluent Greek but loses the thread, and will state false things confidently. Do not use it for facts or advice.
Trained on web text, with the biases of web text; no safety tuning.
Greek only, closed alphabet: Latin letters, most symbols and emoji are silently dropped by the tokenizer.
1,024-token context.

Family

tosodoulikoGPT (10M): odychlapanis/tosodouliko-gpt.
tosodouliGPT (110M): this model.

Code (training, tokenizer, evaluation) will be published on GitHub.

Citation

bibtex

@misc{chlapanis2026tosodouligpt,  author       = {Chlapanis, Odysseas S.},  title        = {tosodouliGPT: a 110M-parameter Greek language model trained from scratch},  year         = {2026},  publisher    = {Hugging Face},  howpublished = {\url{https://huggingface.co/odychlapanis/tosodouli-gpt}}}

Acknowledgements

Training data: FineWeb-2 (Hugging Face, ODC-BY).
Architecture template: nanoGPT (Andrej Karpathy).
Teaching approach: Build a Large Language Model (From Scratch) (Sebastian Raschka).
Evaluation: MultiBLiMP (Jumelet et al.); Greek ARC and HellaSwag translations and FLORES-200 Greek (ILSP / Athena Research Center); FLORES-200 (NLLB Team, Meta AI).

Licence

MIT. The training data (FineWeb-2) is ODC-BY and is not redistributed.

Author: Odysseas S. Chlapanis.

tosodouli-gpt

README

Samples

Use

Model

Training

Evaluation

Limitations

Family

Citation

Acknowledgements

Licence

Explore FriendliAI today

README

Samples

Use

Model

Training

Evaluation

Limitations

Family

Citation

Acknowledgements

Licence