Results on the CAMÕES benchmark (WER %, from Table III)
Table with columns: Model, RS, BN, T/L, CS, SI, EP Avg., AAP, BP| Model | RS | BN | T/L | CS | SI | EP Avg. | AAP | BP |
|---|
| This model (WhisperLv3, fine-tuned) | 7.2 | 4.6 | 13.6 | 14.9 | 43.2 | 16.7 | 101.4 | 28.2 |
| Baseline: original Whisper large-v3, no fine-tuning | 32.4 | 7.9 | 15.4 | 18.3 | 49.0 | 24.6 | 29.9 | 25.8 |
| Baseline: original Whisper large-v3 + WhisperX, no fine-tuning | 16.4 | 8.2 | 16.6 | 15.3 | 39.3 | 19.2 | 29.0 | 24.6 |
The baselines are the public openai/whisper-large-v3 used as released (zero-shot rows of Table III), with
plain Whisper decoding and with WhisperX (voice-activity detection before transcription), respectively.
EP domains: RS = read speech, BN = broadcast news, T/L = talks/lectures, CS = conversational speech,
SI = sociolinguistic interviews. EP Avg. gives each domain equal weight. AAP = African and Asian Portuguese,
BP = Brazilian Portuguese. All results use the CAMÕES text normaliser (lower-case, no punctuation, numbers
spelled out). Leaderboard and test sets: inesc-id/camoes_asr.
Model details
- Base model:
openai/whisper-large-v3 (1.55B parameters, all fine-tuned)
- Training data: EP-425, 425 h of European Portuguese from 14 corpora (Section III-A)
- Fine-tuning: 10 epochs, batch size 64, gradient accumulation 4, learning rate 1e-5, 10% warm-up (Section V)
- Decoding: standard Whisper decoding, without voice-activity detection
- Format: 🤗 Transformers (
safetensors)
Usage
import torchfrom transformers import pipeline asr = pipeline("automatic-speech-recognition", model="inesc-id/WhisperLv3-FT", torch_dtype=torch.float16, device="cuda:0")print(asr("audio.wav", generate_kwargs={"language": "portuguese", "task": "transcribe"})["text"])
For recordings longer than 30 s, add chunk_length_s=30 to pipeline(...), or use
WhisperLv3-EP-X.
Pre-Acordo Ortográfico. All CAMÕES models are trained on transcripts written in the spelling from
before the 1990 Orthographic Agreement (pré-Acordo), e.g. projecto, director, actual.
Transcriptions are normalised like the training transcripts: lower-case, no punctuation, and numbers
written out in words.
Other CAMÕES models
Table with columns: Model, Paper row (Table III), Training data, EP Avg., BP| Model | Paper row (Table III) | Training data | EP Avg. | BP |
|---|
| WhisperLv3-EP-X | WhisperLv3-X, fine-tuned | EP-425 | 12.5 | 27.2 |
| WhisperLv3-FT (this model) | WhisperLv3, fine-tuned | EP-425 | 16.7 | 28.2 |
| EBranch-w2vBERT2-EP | EBranch-w2vBERT2, trained from scratch | EP-425 | 14.4 | 42.4 |
| WhisperLv3-X-PT-All | WhisperLv3-X, PT-All | EP-425 + BP-417 + AAP-8 | 13.0 | 18.8 |
Citation
@inproceedings{camoes, title = {{CAMÕES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguese}}, author = {Carvalho, Carlos and Teixeira, Francisco and Botelho, Catarina and Pompili, Anna and Solera-Ure{\~n}a, Rub{\'e}n and Paulo, S{\'e}rgio and Juli{\~a}o, Mariana and Rolland, Thomas and Mendon{\c{c}}a, John and Pereira, Diogo and Trancoso, Isabel and Abad, Alberto}, booktitle = {Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)}, year = {2025}}