AliacAI

whisper-base-el-en

README

License: apache-2.0

Summary

Greek: 28.65% lexical WER on the held-out FLEURS Greek test split, against 62.50% for stock Whisper Base and 35.11% for stock Whisper Small, which has 3.3 times as many parameters.
English by Greek speakers, language detected between Greek and English: on read English by 11 Greek and Cypriot speakers (Speech Accent Archive), the model decodes every speaker as English and reaches 11.86% WER, while every stock Whisper model writes Greek for one to three of the speakers (stock Whisper Base 67.33%); with 11 speakers, no stock model is separably worse. On the milder accents of Greek and Cypriot members of the European Parliament, 21.47% against 59.18% for stock Whisper Base; stock Whisper Small (27.29%) is not separable, and the large models are better.
English with the language forced: worse than stock Whisper Base, by 4.55 points on FLEURS English test, by 3.1 to 4.1 points on English by Greek speakers, and by 3.49 points on the mean of nine Open ASR Leaderboard sets (15.58% against 12.09%).
Training cost: 65 minutes on one RTX 4090.

Model details

Table with columns: Property, Value
Property	Value
Architecture	Whisper encoder-decoder transformer
Parameters	72,593,920
Base model	`openai/whisper-base`, revision `e37978b9`
Languages	Greek (`el`), English (`en`)
Training data	179.2 h Greek, 90.0 h English
Input	16 kHz mono audio, up to 30 s per window
Decoding	Greedy, no timestamps
License	Apache-2.0

Files

Table with columns: File, Format, Use
File	Format	Use
`model.safetensors` and configs	Transformers, fp32	Python, GPU or CPU
`ggml-model-q8_0.bin`	whisper.cpp, q8_0	Laptop and desktop CPU
`whisper-base-el-en-report.pdf`	PDF	Technical report

Evaluation

All models were decoded greedily with identical settings, and the models compared on a test set were decoded on the same GPU. Intervals are 95% bootstrap intervals over whole FLEURS sentences, YODAS videos, or VoxPopuli speakers. Greek scores keep accents, so a missing tonos counts as an error.

Held-out test split

Lexical WER (%) on the frozen test split of the training corpus. YODAS references are uploader captions from a few videos, so YODAS measures agreement with captions.

Table with columns: Model, Params (M), Greek FLEURS, English FLEURS, Greek YODAS, English YODAS
Model	Params (M)	Greek FLEURS	English FLEURS	Greek YODAS	English YODAS
Whisper Tiny	37.8	85.35 [78.1, 94.6]	14.45 [13.3, 15.6]	181.07 [138.9, 276.7]	45.55 [23.6, 60.9]
Whisper Base	72.6	62.50 [58.9, 67.1]	11.67 [9.5, 15.1]	183.97 [157.7, 224.7]	38.04 [21.4, 49.4]
Base el-en (ours)	72.6	28.65 [27.0, 30.5]	16.21 [14.0, 19.6]	37.62 [31.9, 49.9]	44.06 [22.5, 66.3]
Whisper Small	241.7	35.11 [33.0, 37.7]	6.72 [6.0, 7.5]	134.72 [111.5, 201.7]	29.54 [17.3, 38.1]
Whisper Medium	763.9	20.77 [19.1, 22.8]	5.20 [4.6, 5.8]	110.52 [97.5, 167.1]	29.16 [19.4, 35.8]
Whisper large-v3-turbo	808.9	13.25 [12.2, 14.4]	4.95 [4.4, 5.5]	37.57 [34.2, 47.1]	37.92 [20.2, 78.3]
Whisper large-v3	1543.5	11.45 [10.5, 12.5]	4.78 [4.2, 5.3]	37.57 [33.7, 49.6]	28.34 [19.2, 34.6]

English by Greek speakers

Two test sets, neither used in training. WER (%) after Whisper's English normalizer.

Parliamentary English (VoxPopuli). English by Greek and Cypriot members of the European Parliament (504 clips, 24 speakers), a native control by UK and Irish members (508 clips, 140 speakers), and 40 clips for each of 15 other accents (590 clips), with English forced. The gap is the Greek pool minus the native control. The accent is mild: no stock model loses more than 1.83 points on the Greek pool against the native control.

Table with columns: Model, Greek and Cypriot, UK and Irish, Other accents, Gap
Model	Greek and Cypriot	UK and Irish	Other accents	Gap
Whisper Tiny	13.71 [11.2, 18.7]	14.93 [9.9, 23.6]	22.79 [19.1, 28.4]	-1.22 [-9.7, 6.2]
Whisper Base	9.80 [8.6, 11.0]	7.98 [7.3, 8.7]	20.51 [17.3, 25.0]	1.83 [0.4, 3.3]
Base el-en (ours)	12.89 [11.9, 13.7]	11.63 [10.7, 12.5]	18.17 [16.8, 19.5]	1.25 [-0.1, 2.5]
Whisper Small	8.21 [7.1, 9.4]	6.40 [5.7, 7.1]	16.83 [15.4, 18.3]	1.80 [0.4, 3.3]
Whisper Medium	7.46 [6.4, 8.7]	5.87 [5.3, 6.5]	18.31 [15.7, 21.5]	1.59 [0.3, 3.0]
Whisper large-v3-turbo	11.46 [8.6, 15.1]	10.62 [8.5, 13.0]	28.09 [25.2, 31.0]	0.83 [-2.9, 5.2]
Whisper large-v3	9.51 [7.4, 12.2]	9.21 [7.7, 11.0]	21.79 [19.7, 24.1]	0.30 [-2.5, 3.5]

With the language forced to English, Base el-en is less accurate than stock Whisper Base on every pool, by 3.08 points on the Greek pool and 3.66 points on the native control, so the loss is not specific to the accent.

Read English (Speech Accent Archive). 11 Greek and Cypriot speakers who learned English at school and 30 native speakers from the USA, the UK, and Ireland each read the same 69-word paragraph. Each recording is cut once at a pause to fit Whisper's 30-second window. Forced: English forced. Detected: detection restricted to Greek and English, with the number of Greek speakers decoded as Greek.

Table with columns: Model, Forced: Greek, Forced: native, Forced: gap, Detected: Greek, Greek speakers decoded as Greek
Model	Forced: Greek	Forced: native	Forced: gap	Detected: Greek	Greek speakers decoded as Greek
Whisper Tiny	8.56 [4.5, 14.0]	5.80 [3.7, 8.3]	2.77 [-1.9, 8.4]	96.05 [7.2, 229.4]	2 of 11
Whisper Base	7.77 [4.7, 12.3]	4.25 [2.4, 6.4]	3.52 [-0.3, 8.0]	67.33 [9.6, 163.5]	3 of 11
Base el-en (ours)	11.86 [8.0, 16.3]	8.16 [5.9, 10.7]	3.69 [-0.9, 8.7]	11.86 [8.0, 16.3]	0 of 11
Whisper Small	4.48 [1.6, 8.6]	1.69 [0.8, 2.8]	2.79 [-0.5, 6.7]	21.48 [3.6, 45.7]	2 of 11
Whisper Medium	3.03 [0.8, 6.2]	1.35 [0.6, 2.3]	1.68 [-0.8, 4.7]	33.73 [9.7, 70.1]	3 of 11
Whisper large-v3-turbo	2.24 [0.4, 5.0]	1.16 [0.4, 2.1]	1.08 [-1.1, 3.8]	11.33 [0.8, 29.6]	1 of 11
Whisper large-v3	2.37 [0.4, 4.7]	1.50 [0.5, 2.8]	0.87 [-1.5, 3.5]	22.40 [4.1, 46.5]	3 of 11

Base el-en decodes every Greek speaker as English. Every stock model decodes one to three of them as Greek and writes Greek. With 11 speakers, no difference against Base el-en is statistically separable, and under forced English every stock model is more accurate.

Language detection on parliamentary English

WER (%) and the share of clips detected as a language other than English (%), with detection among all of Whisper's languages and restricted to Greek and English.

Table with columns: Model, All: Greek WER, All: Greek other, All: native WER, All: native other, el/en: Greek WER, el/en: Greek other, el/en: native WER, el/en: native other
Model	All: Greek WER	All: Greek other	All: native WER	All: native other	el/en: Greek WER	el/en: Greek other	el/en: native WER	el/en: native other
Whisper Tiny	161.69	35.1	312.89	40.2	119.53	26.0	14.93	0.0
Whisper Base	103.24	21.6	236.89	40.9	59.18	14.7	7.98	0.0
Base el-en (ours)	20.82	20.8	14.45	0.4	21.47	21.8	14.45	0.4
Whisper Small	41.23	21.6	284.55	62.2	27.29	18.7	6.40	0.0
Whisper Medium	85.81	46.0	180.38	65.9	41.88	33.3	5.87	0.0
Whisper large-v3-turbo	15.98	7.7	13.32	4.3	12.80	1.8	10.62	0.0
Whisper large-v3	10.06	0.8	20.10	9.8	9.89	0.4	9.21	0.0

With detection restricted to Greek and English, Base el-en is 37.71 points [17.9, 68.9] more accurate than stock Whisper Base and 20.40 points [10.4, 31.2] more accurate than stock Whisper Medium on the Greek pool. Stock Small is not separable from it (5.82 points [-2.8, 21.5]), and large-v3-turbo and large-v3 are more accurate. Base el-en detects Greek on 110 of 504 clips, but 69 of those outputs contain no Greek letters; all 74 stock Base outputs on clips it detects as Greek are written in Greek.

English on the Open ASR Leaderboard suite

Scored privately with the leaderboard's own script and normalizer. Mean WER over the nine public sets: 12.09% for stock Whisper Base, 15.58% for Base el-en. Fine-tuning made English worse on every set.

How to run

Transformers

python

from transformers import pipeline asr = pipeline("automatic-speech-recognition", model="AliacAI/whisper-base-el-en")result = asr("speech.wav", generate_kwargs={"language": "el", "task": "transcribe"})print(result["text"])

Set "language": "en" for English, or leave it out to detect the language.

whisper.cpp

bash

whisper-server --model ggml-model-q8_0.bin --language auto --no-timestamps \  --vad --vad-model ggml-silero-v6.2.0.bin \  --vad-speech-pad-ms 300 --vad-min-silence-duration-ms 500

Voice activity detection removes phrases that the model otherwise writes during silence. Keep the 300 ms speech padding: the default 30 ms clips final words.

Training

Full fine-tuning for 8,000 updates of 64 clips, peak learning rate 2.5e-5 with cosine decay, SpecAugment time and frequency masking, dropout zero, bf16, seed 42; the checkpoint at step 6,000 was selected on FLEURS validation. Training draws are 70% Greek and 30% English from VoxPopuli Greek with MOSEL machine transcripts, YODAS, Common Voice 27.0, FLEURS, VoxForge, the author's own dictation recordings, and synthetic non-speech clips.

Limitations

English on native speech is less accurate than stock Whisper Base.
Greedy decoding can repeat a phrase until the length limit.
The model deletes the final word of 7.4% of Greek FLEURS validation clips.
Without voice activity detection, silence can produce short invented phrases.
The only Greek-accented English in training comes from one speaker, the author. The accent test sets hold mildly accented parliamentary speech (24 speakers) and one read paragraph from each of 11 clearly accented speakers; spontaneous, strongly accented speech was not tested.
Mixed Greek and English within one utterance, English technical terms inside Greek speech, timestamps, telephone audio, and children's speech were not evaluated.
Every result comes from one training run with one seed.

License

The weights are released under Apache-2.0, the license of the base model openai/whisper-base. Training sources carry their own licenses: CC0-1.0 (Common Voice, VoxPopuli audio), CC-BY-4.0 (FLEURS, MOSEL transcripts), CC-BY-3.0 (YODAS), and GPL-3.0 (VoxForge).

Citation

bibtex

@techreport{kostopoulos2026whisperbaseelen,  title       = {Whisper Base el-en: Greek and English Speech Recognition with 72.6 Million Parameters},  author      = {Kostopoulos, Yiorgos},  institution = {Aliac Research},  year        = {2026},  month       = oct,  type        = {Technical report}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

AliacAI

Model Tree

Base

openai/whisper-base

Fine-tuned

this model

Input Modalities

Audio

Output Modalities

Text

Supported Functionality