Quick start
Transcript with speakers (recommended)
hf download eek/whisper-turbo-ro-en transcribe.py requirements.txt requirements-speakers.txt --local-dir vorbacd vorbapip install -r requirements.txt # transcripts (faster-whisper)pip install -r requirements-speakers.txt # only for --speakers (PyTorch + NVIDIA NeMo) python transcribe.py interview.m4a # plain transcriptpython transcribe.py interview.m4a --speakers # [00:01.20] S1: ...python transcribe.py interview.m4a --speakers --srt out.srt --json out.jsonpython transcribe.py clip.mp4 --language auto --mode cover # detect language; dense speech
transcribe.py reads any audio or video file FFmpeg can decode, runs on a GPU when one is
available (otherwise on the CPU, slower), and downloads the models on first use. Options:
Table with columns: Option, Meaning| Option | Meaning |
|---|
--language ro|en|auto | force the language (default ro; Romanian also handles Romglish) or detect it |
--mode native (default) | Whisper's own 30 s seeking. Best for far-field speech, children, conversations |
--mode cover | windows that end in pauses, each prompted with the previous window's text. Best for clean, dense speech (videos, podcasts), and silent on music |
--speakers | add the diarization model; each word gets a speaker (needs NVIDIA NeMo, see requirements) |
--json, --srt | write words with times and speakers, or subtitles |
faster-whisper directly (long audio, fastest)
The folder ctranslate2-int8/ is a CTranslate2 int8 conversion.
from faster_whisper import WhisperModelfrom huggingface_hub import snapshot_download path = snapshot_download("eek/whisper-turbo-ro-en", allow_patterns=["ctranslate2-int8/*"]) + "/ctranslate2-int8"model = WhisperModel(path, device="cuda", compute_type="int8_float16") # CPU: device="cpu", compute_type="int8"segments, info = model.transcribe( "audio.wav", language="ro", beam_size=5, condition_on_previous_text=False, word_timestamps=True)for s in segments: print(f"[{s.start:6.2f} → {s.end:6.2f}] {s.text}")
Keep condition_on_previous_text=False: Whisper's default of prompting each window with the
previous text makes this model (like most fine-tunes) loop or skip on long recordings.
The repository root holds the model in Hugging Face format (fp16 safetensors).
import torchfrom transformers import pipeline asr = pipeline("automatic-speech-recognition", model="eek/whisper-turbo-ro-en", dtype=torch.float16, device="cuda:0")print(asr("clip.wav", generate_kwargs={"language": "ro", "task": "transcribe"})["text"])
For audio longer than 30 seconds prefer faster-whisper or transcribe.py.
Output conventions
- Verbatim, as a careful transcriber would write it: casing, punctuation, colloquial spellings
kept (
acuma, aicea, anu), numbers usually as digits.
[S1] … [S2] … marks speaker changes inside one 30 s window only. The labels restart in every
window, so strip them (re.sub(r"\[S\d+\]\s*", "", text)) or use the diarization model for
recording-level speakers. transcribe.py does the latter.- The saved generation config un-suppresses the
[, [ and ] tokens (Whisper suppresses
them by default); keep it if you copy the model elsewhere, or the tags cannot be produced.
- With
suppress_blank=False (as transcribe.py --mode cover decodes) the model returns nothing
on music and silence instead of Whisper's classic "Mulțumesc pentru vizionare!".
Benchmarks
Word error rate (%), lower is better, normalized with Gladia's LLM_MARKDOWN_PROTECTED_25
preset plus a Romanian plug-in (diacritics folded, colloquial spellings kept distinct, spoken
numbers as digits).
Compared with other Romanian models
- Best average over eight public Romanian test sets: 8.0. The closest third-party model,
jackrabbit-110m-ro with its 4-gram LM, averages 9.5; OpenAI's Whisper large-v3 averages 11.4.
- Largest lead on spontaneous and conversational speech: Romanian films 13.8 (best
third-party model: 19.4), spontaneous online videos 13.6 (16.5) and a far-field household
conversation with children 17.5 (26.2).
- Romanian and English in one model: FLEURS English 5.4, near stock Whisper's 4.6. The
Romanian specialists that come closest on Romanian transcribe Romanian only.
- Read speech and parliament remain jackrabbit's strength: Common Voice 1.9 vs 2.7, VoxPopuli
5.7 vs 7.8.
Table with columns: #, Model, Avg. (8 RO sets), FLEURS ro, FLEURS en, Films, Spontaneous, Common Voice, Household, VoxPopuli, Podcasts, Audiobooks, Stories| # | Model | Avg. (8 RO sets) | FLEURS ro | FLEURS en | Films | Spontaneous | Common Voice | Household | VoxPopuli | Podcasts | Audiobooks | Stories |
|---|
| 1 | This model | 8.0 | 6.1 | 5.4 | |
Every Romanian-capable model we could run, on every public Romanian test set with human
transcripts that can be downloaded without an access request, plus the household conversation
described below. Greedy decoding except where an n-gram LM is named; the language is forced to
Romanian (English for FLEURS en) where a model takes one. "Avg." is the unweighted mean of the
eight public Romanian sets (FLEURS en and the household conversation are not in it). Clips longer
than 30 s are left out (6 in VoxPopuli, 27 in USPDATRO) because Whisper decodes one 30-second
window. Best comparable score in each column in bold. † The model's card says it was trained on
this set or on an overlapping Common Voice release, so that score is not comparable.
Table with columns: Set, Speech, Clips, Hours| Set | Speech | Clips | Hours |
|---|
| FLEURS test | read | 883 | 2.5 |
| RO-N3WS films | Romanian films, conversational | 4,723 | 8.3 |
| USPDATRO ("Spontaneous") | spontaneous and dialectal, from online videos | 2,610 |
- jackrabbit (training data not disclosed) and our 110M model are strongest on read speech and
parliament but fall behind on spontaneous speech (films, USPDATRO, the household
conversation), and they transcribe Romanian only (FLEURS en near 100%).
- upb-nlp/ro-wav2vec2 and gigant/romanian-wav2vec2 ship an n-gram LM that was not used here, so
their published figures are lower.
- FraPiz's model loops on the household recording (142%).
- Not included: Echo (gated), the SpeD RSC/SSC/CDEP sets and SWARA (by request), and models that
need another runtime (moondream/parakeet-ultra).
Against stock Whisper turbo
"Private" sets are held-out conversational recordings that were never trained on; their
references are machine transcripts unless marked human.
Table with columns: Test set, Reference, Whisper turbo (stock), This model| Test set | Reference | Whisper turbo (stock) | This model |
|---|
| FLEURS Romanian test | human | 9.0 | 6.11 |
| FLEURS English test | human | 4.6 | 5.40 |
| Private Romanian conversational set, 19 h | machine | 10.6 | 3.91 |
| Private Romanian set from newer recordings, 10 h | machine | 9.8 |
For scale: the machine references and the independent captions disagree with each other by
11.9%, and the best commercial system we tried scores 4.2 on the household conversation. The
stock figure for that recording uses 28 s chunks (this model: 17.5 on the same chunks, 13.7 with
native decoding).
Whole recordings of 30–600 s, --mode cover: Romanian 4.12, English 1.74. Speaker-attributed
WER (cpWER) on multi-speaker recordings with the companion diarization model, --mode cover:
15.5 (Romanian) and 15.1 (English) with the current diarizer, and 14.3 and 14.8 with its revision
ft-d1, which is better on these recordings but much worse on overlapping speech. On 132
Romanian music-or-silence clips the model outputs nothing in cover mode.
Languages it was not trained on
FLEURS test with the correct language token forced. Use stock Whisper for these:
Table with columns: Language, Whisper turbo (stock), This model| Language | Whisper turbo (stock) | This model |
|---|
| German | 4.56 | 9.25 |
| Spanish | 2.86 | 14.19 |
| French | 5.11 | 31.20 |
| Italian | 3.77 | 20.43 |
| Portuguese | 4.16 | 57.58 |
| Polish | |
Training
Fine-tuned on a private dataset of Romanian and English conversational speech, in five stages,
each continuing from the previous best (full fine-tune, bf16, AdamW, cosine schedule):
- Romanian segments plus about 10% English, lr 1e-5, one epoch.
- More and longer recordings, packed to 28 s, lr 5e-6.
- Conversation windows of up to 28 s with
[S#] speaker turns, timestamps on half the windows,
the previous window's text as prompt on half, speed perturbation, SpecAugment, and
no-speech windows with empty targets; lr 5e-6.
- A half pass at lr 3e-6 that also grows windows into their quiet surroundings and splices in
no-speech audio, so windows that open with music are transcribed correctly.
- This model: one more full pass with stage 4's recipe at lr 5e-6 over a larger version of the
dataset (about 45% more Romanian).
About 10% English throughout kept English from degrading.
Limitations
- Children, far-field microphones and overlapping speakers are much harder: on the household
conversation the error is about three times that of the best commercial system.
- The training references were not human-verified, so the model inherits some of their
conventions and mistakes.
- Read-speech English is slightly worse than stock Whisper (5.4 vs 4.6 on FLEURS).
- Use stock Whisper for other languages. Languages other than Romanian and English were
not trained and degraded badly, even with the correct language token: mean FLEURS WER over
German, Spanish, French, Italian, Portuguese, Polish, Hungarian, Russian, Ukrainian and
Turkish is 6.1 for stock Whisper and 54.9 for this model, and output drifts into Romanian
spelling.
- In
cover mode, English music-only windows still produce a few words now and then.
Licence
MIT, the same licence as the base model (OpenAI Whisper): use, modify and redistribute it
freely, commercial use included, keeping the notice in LLM_MARKDOWN_PROTECTED_32. This covers the
weights, the CTranslate2 conversion and transcribe.py. The fine-tuning data is private and is
not distributed.
