Results
Table with columns: Model, Raw WER, Normalized WER, Test clips| Model | Raw WER | Normalized WER | Test clips |
|---|
| whisper-base fine-tuned | 105.7% | 94.1% | 300 |
| whisper-base zero-shot | 107.6% | 134.7% | 300 |
| whisper-small fine-tuned | 46.7% | 38.9% | 300 |
| whisper-small zero-shot | 110.1% | 100.6% | 300 |
Zero-shot Whisper barely functions on Maltese (≈1 h of it in the
original training mix); the WER delta above is what a few hours of
community-donated Common Voice speech and a laptop fine-tune buy. WER
computed on the official Common Voice test split (speaker-disjoint from
training data — note the training split draws on only 6
speakers, which limits generalization; see Limitations). "Normalized"
applies BasicTextNormalizer (lowercase, strip punctuation) to both
reference and hypothesis.
Data provenance
- Source: Mozilla Common Voice Scripted Speech 26.0,
Maltese (mt), obtained via the
Mozilla Data Collective
(since October 2025 the exclusive distribution channel for Common Voice
datasets — the former
mozilla-foundation/common_voice_* datasets on this
Hub are empty shells).
- License: the underlying speech data is CC0.
- Used here: 3000 training clips (~3.77 h,
6 speakers), 200 dev clips, 300 test clips,
official splits, clips 1–15 s.
- The prepared audio dataset is deliberately not re-uploaded to this Hub;
get the data from the Mozilla Data Collective.
Training
Standard Hugging Face Seq2SeqTrainer recipe
(fine-tune-whisper):
batch 4 × grad-accum 4, lr 1e-05, warmup 50,
1000 steps, fp32 (MPS), gradient checkpointing, greedy decoding,
best-checkpoint-by-WER selection.
Limitations
- Single low-resource language, ~3.77 h of scripted (read) speech
from only 6 speakers (the corpus's many one-clip
contributors are concentrated in the held-out splits) — expect degradation
on spontaneous/conversational audio and unseen voices.
- Small test set (300 clips); WER has meaningful variance.
- No punctuation/casing guarantees; laptop-scale training run, not a
production model.
Usage
from transformers import pipeline
asr = pipeline("automatic-speech-recognition", model="titaniumbones/whisper-small-mt-commonvoice")
print(asr("clip.mp3", generate_kwargs={"language": "maltese"}))