duhow

veucat

README

License: apache-2.0

Key results

All reported WER/CER values are percentages; lower is better. RTF is the inference-time-to-audio-duration ratio measured in the evaluation setup (lower is faster).

Mozilla Common Voice 27.0 — Catalan

Table with columns: Split, Clips, Audio hours, WER ↓, CER ↓, RTF ↓
Split	Clips	Audio hours	WER ↓	CER ↓	RTF ↓
dev	16,416	27.84	8.77%	3.27%	0.003
test	16,415	27.64	10.88%	4.48%	0.003

Mozilla Common Voice 27.0 — Spanish

Table with columns: Split, Clips, Audio hours, WER ↓, CER ↓, RTF ↓
Split	Clips	Audio hours	WER ↓	CER ↓	RTF ↓
dev	15,907	26.87	13.24%	4.96%	0.003
test	15,907	26.97	15.58%	6.24%	0.003

Additional Catalan evaluation datasets

The following datasets were not used as training sources. Their results are useful as an indication of domain transfer beyond Common Voice; they are not directly comparable to Common Voice because the recording conditions and text distributions differ.

Table with columns: Dataset, Clips, Audio hours, WER ↓, CER ↓, RTF ↓
Dataset	Clips	Audio hours	WER ↓	CER ↓	RTF ↓
FestCat — Projecte Aina, trimmed and denoised	12,435	22.91	22.88%	15.91%	0.005
OpenSLR69 — Projecte Aina, Catalan, trimmed and denoised	4,240	4.99	13.46%	5.42%	0.006

The number of evaluated recordings matches the published Projecte Aina variants above. The external evaluations are reported as supplied; their preprocessing and normalization should be kept consistent when comparing other models.

Model architecture

The implementation is based on transformers.WhisperForConditionalGeneration, instantiated with a new WhisperConfig and randomly initialized weights.

Table with columns: Component, Configuration
Component	Configuration
Architecture	Whisper Base-style encoder–decoder Transformer
Encoder / decoder	6 Transformer layers each
Hidden dimension	512
Attention heads	8 per layer
Feed-forward dimension	2,048
Acoustic input	80-bin log-Mel spectrogram
Audio sampling rate	16,000 Hz, mono
Audio window	Up to 30 seconds (3,000 Mel frames; 1,500 encoder positions)
FFT / hop length	400 / 160 samples
Decoder context	Up to 448 positions in the architecture
Tokenizer	Custom ByteLevel BPE trained from Common Voice Catalan and Spanish text
Vocabulary	24,000 BPE text tokens; 25,608 tokens total including special/reserved tokens
Training task	Language-conditioned transcription without timestamps

Tokenizer compatibility: The custom vocabulary reserves the canonical Whisper sequence of 99 language token IDs, task/control tokens, and 1,501 timestamp tokens. These are retained for tokenizer/converter compatibility; only Catalan and Spanish were trained. Reserved language tokens do not imply multilingual recognition in the other languages, and reserved timestamp tokens were not trained to produce reliable timestamps.

Unlike standard OpenAI Whisper checkpoints, this model has a different vocabulary and token-ID mapping. Always use the tokenizer distributed with this model; do not substitute a tokenizer from openai/whisper-*.

Training data and preparation

The only training speech dataset was Mozilla Common Voice Scripted Speech 27.0 (corpus release cv-corpus-27.0-2026-09-11), using its Catalan (ca) and Spanish (es) train.tsv partitions.

Data preparation followed the project's prepare.py and train.py scripts:

Read the official language-specific train.tsv, dev.tsv, and test.tsv manifests and resolve audio clip paths.
Normalize transcript text to Unicode NFC and normalize whitespace; reject invalid, missing, or empty examples.
Prioritize evaluation partitions when removing duplicate audio paths across the splits within a language.
Decode audio with FFmpeg, convert to mono 16 kHz floating-point PCM, and cap each training example at 30 seconds.
Extract the standard 80-bin Whisper log-Mel features.
Train the ByteLevel BPE tokenizer from scratch, using reservoir samples of up to 250,000 training sentences per language (vocab_size=24000, min_frequency=5, max_token_length=24). Add Whisper-compatible special tokens after the text vocabulary.

The training manifest's dataset wrapper forms groups of five logical samples: four Catalan and one Spanish. This gives an exact 80:20 presentation ratio per five samples. It may repeat samples when one language's underlying manifest is exhausted; 80:20 is a sampling ratio, not a statement about the raw number of files or hours per language.

The training target for each clip is:

text

<|startoftranscript|> <|ca|> or <|es|> <|transcribe|> <|notimestamps|> TEXT <|endoftext|>

The decoder predicts the language marker and transcription task tokens as part of the sequence. Transcript text is limited to 160 BPE tokens per training example. No timestamp alignment supervision or translation supervision was used.

Training procedure

Training was performed in three successive phases using Hugging Face Trainer. A new continuation phase loads the preceding phase's learned model weights, then starts a fresh AdamW optimizer and cosine learning-rate scheduler. If a phase is interrupted, its checkpoint is resumed with the saved optimizer and scheduler state instead of restarting them.

Table with columns: Phase, Updates in phase, Cumulative updates, Peak learning rate, Initial weights
Phase	Updates in phase	Cumulative updates	Peak learning rate	Initial weights
v2 — initial training	300,000	300,000	`2e-4`	Random initialization
v3 — continuation	50,000	350,000	`2e-5`	v2 (300K)
v4 — continuation	150,000	500,000	`1e-5`	v3 (350K)

Training parameters shared across the phases:

Table with columns: Hyperparameter, Value
Hyperparameter	Value
Framework	PyTorch + Hugging Face Transformers `Trainer`
Optimizer	AdamW (`adamw_torch`)
Scheduler	Cosine decay to zero; 5% warmup in each phase
Weight decay	`0.01`
Per-device training batch size	`1` in the reported single-GPU configuration
Gradient accumulation	`32` optimizer microbatches
Effective batch size	`32` examples per optimizer step (single GPU)
Max gradient norm	`1.0`
Dropout	`0.1`
SpecAugment	Enabled (`mask_time_prob=0.05`, `mask_time_length=10`)
Gradient checkpointing	Enabled
Random seed	`42`
Mixed precision	BF16 when supported by CUDA, otherwise FP16 on CUDA
Language sampling	80% Catalan / 20% Spanish throughout

The project used an NVIDIA RTX 5090 Laptop GPU (24 GB VRAM) for training, with a CUDA-enabled PyTorch environment. Exact throughput and training duration depend on hardware and data I/O.

Validation during training: Teacher-forced eval_loss was monitored on a fixed random sample of 256 Common Voice dev clips per language (512 total). This small internal validation set is different from the full-set WER/CER evaluations reported above. The final v4 checkpoint continued to improve Catalan WER despite a nearly flat/slightly higher teacher-forced validation loss.

Evaluation methodology

Common Voice: evaluated against the full dev.tsv and test.tsv partitions for each language of v27.0, rather than the 512-clip training-time validation subset.
FestCat and OpenSLR69: separately evaluated Catalan datasets used to assess out-of-domain performance. Their published Hugging Face dataset splits are labeled train; that label refers to the dataset packaging, not their use in training this model.
Language: supplied explicitly as ca or es, with transcription (transcribe) and timestamps disabled.
Decoding and inference: the reported Common Voice evaluator used batched Transformers generation with beam size 1 (greedy decoding), CUDA/FP16, and a fixed maximum decoder length.
WER/CER: computed with jiwer after Unicode NFC normalization, lowercasing, normalizing apostrophe variants and Catalan middle-dot spelling, removing most punctuation, and collapsing whitespace. Apostrophes, hyphens, accents, and middle dots are preserved where relevant.
RTF: measured from the evaluation implementation and batching configuration; it is not a hardware-independent benchmark.

The numbers above describe the 500K-step checkpoint, not necessarily other checkpoints or quantized variants.

Usage with Transformers

Use the matching custom tokenizer and an 80-Mel feature extractor. The example below is for a short clip of up to 30 seconds; longer audio needs an explicit segmentation/long-form strategy, which has not been validated for this no-timestamp training setup.

bash

pip install torch 'transformers>=4.50,<5' librosa

python

import torchimport librosafrom transformers import (    PreTrainedTokenizerFast,    WhisperFeatureExtractor,    WhisperForConditionalGeneration,) MODEL_ID = "YOUR_USERNAME/YOUR_MODEL_REPO"  # Replace with this model's Hub IDLANGUAGE = "ca"  # Or "es"AUDIO_FILE = "example.wav" assert LANGUAGE in {"ca", "es"}device = "cuda" if torch.cuda.is_available() else "cpu"dtype = torch.float16 if device == "cuda" else torch.float32 tokenizer = PreTrainedTokenizerFast.from_pretrained(MODEL_ID)model = WhisperForConditionalGeneration.from_pretrained(MODEL_ID)model = model.to(device=device, dtype=dtype).eval() # Use 80 Mel bins: do not load the frontend from Whisper large-v3 (128 Mel).extractor = WhisperFeatureExtractor(    feature_size=80,    sampling_rate=16000,    chunk_length=30,    n_fft=400,    hop_length=160,) # Explicit CA/ES language selection and no timestamps.model.generation_config.is_multilingual = Truemodel.generation_config.lang_to_id = {    f"<|{lang}|>": tokenizer.convert_tokens_to_ids(f"<|{lang}|>")    for lang in ("ca", "es")}model.generation_config.task_to_id = {    "transcribe": tokenizer.convert_tokens_to_ids("<|transcribe|>"),    "translate": tokenizer.convert_tokens_to_ids("<|translate|>"),}model.generation_config.no_timestamps_token_id = tokenizer.convert_tokens_to_ids("<|notimestamps|>")model.generation_config.forced_decoder_ids = None audio, _ = librosa.load(AUDIO_FILE, sr=16000, mono=True)if len(audio) > 30 * 16000:    raise ValueError("This example supports a maximum of 30 seconds of audio")input_features = extractor(    audio, sampling_rate=16000, return_tensors="pt").input_features.to(device=device, dtype=dtype) with torch.inference_mode():    output_ids = model.generate(        input_features=input_features,        language=LANGUAGE,        task="transcribe",        return_timestamps=False,        max_new_tokens=160,        do_sample=False,        use_model_defaults=False,  # Transformers 4.50+: avoid duplicate suppression processors    ) text = tokenizer.batch_decode(    output_ids, skip_special_tokens=True,    clean_up_tokenization_spaces=False,)[0]print(text.strip())

This example explicitly uses PreTrainedTokenizerFast because the originally exported model may identify its tokenizer in tokenizer_config.json as the generic PreTrainedTokenizer, which can confuse AutoTokenizer. It also constructs the correct 80-Mel extractor rather than depending on the presence of preprocessor_config.json.

Recommended repository files

For a cleaner model distribution, upload the checkpoint weights and their matching tokenizer together with:

text

config.jsonmodel.safetensors               # or the actual sharded weight files

text

generation_config.jsontokenizer.jsontokenizer_config.jsonspecial_tokens_map.jsonpreprocessor_config.json        # recommended; missing in some v4 exportsREADME.md

If your model export lacks preprocessor_config.json, generate it without changing the model's weights:

python

from transformers import WhisperFeatureExtractor WhisperFeatureExtractor(    feature_size=80,    sampling_rate=16000,    chunk_length=30,    n_fft=400,    hop_length=160,).save_pretrained("v4")  # Replace with the local model-export directory

If the tokenizer metadata still declares PreTrainedTokenizer, verify and correct the tokenizer class metadata for Transformers compatibility without retraining or replacing the vocabulary. Validate token IDs and sample outputs before changing published files.

Some Transformers 4.x versions may report a duplicate SuppressTokensLogitsProcessor warning when Whisper's custom suppression configuration is inherited again by the generic generator. This is a generation-configuration warning, not evidence of damaged weights. Do not indiscriminately clear suppress_tokens: this model suppresses unused language/task tokens by design. The example above uses the applicable Transformers compatibility flag.

Intended use and limitations

Intended: Catalan and Spanish transcription for audio up to 30 seconds per chunk, with the language provided explicitly. It can be used as a research checkpoint or as a starting point for domain-specific ASR adaptation.

Not established / not supported by training:

Reliable word- or segment-level timestamps (the timestamp tokens exist but were not trained).
Speech translation, speech synthesis, or recognition of the other reserved Whisper languages.
Robust automatic language identification, multilingual code-switching, or no-speech/VAD behavior without separate validation.
Reliable transcription of noisy conversations, overlapping voices, long recordings, or specialized terminology without task-specific tests.

Read-speech benchmarks should not be treated as guarantees for production performance. In particular, FestCat WER is substantially higher than Common Voice Catalan WER; real-world errors can vary considerably by speaker, microphone, vocabulary, and recording conditions. Transcriptions can contain omissions or substitutions, so human review is appropriate for consequential uses.

Data attribution and licensing

Training data: Mozilla Common Voice Scripted Speech 27.0 (Catalan and Spanish), distributed under CC0-1.0 via the Mozilla Data Collective. Respect the dataset's distribution terms and speaker privacy restrictions. The separate external evaluation sets have their own licenses; see their linked dataset cards (the Projecte Aina OpenSLR69 derivative is CC BY-SA 4.0).

References

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

duhow

Model Tree

Base

this model

Input Modalities

Audio

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Key results

All reported WER/CER values are percentages; lower is better. RTF is the inference-time-to-audio-duration ratio measured in the evaluation setup (lower is faster).

Mozilla Common Voice 27.0 — Catalan

Table with columns: Split, Clips, Audio hours, WER ↓, CER ↓, RTF ↓
Split	Clips	Audio hours	WER ↓	CER ↓	RTF ↓
dev	16,416	27.84	8.77%	3.27%	0.003
test	16,415	27.64	10.88%	4.48%	0.003

Mozilla Common Voice 27.0 — Spanish

Table with columns: Split, Clips, Audio hours, WER ↓, CER ↓, RTF ↓
Split	Clips	Audio hours	WER ↓	CER ↓	RTF ↓
dev	15,907	26.87	13.24%	4.96%	0.003
test	15,907	26.97	15.58%	6.24%	0.003

Additional Catalan evaluation datasets

Table with columns: Dataset, Clips, Audio hours, WER ↓, CER ↓, RTF ↓
Dataset	Clips	Audio hours	WER ↓	CER ↓	RTF ↓
FestCat — Projecte Aina, trimmed and denoised	12,435	22.91	22.88%	15.91%	0.005
OpenSLR69 — Projecte Aina, Catalan, trimmed and denoised	4,240	4.99	13.46%	5.42%	0.006

Model architecture

The implementation is based on transformers.WhisperForConditionalGeneration, instantiated with a new WhisperConfig and randomly initialized weights.

Table with columns: Component, Configuration
Component	Configuration
Architecture	Whisper Base-style encoder–decoder Transformer
Encoder / decoder	6 Transformer layers each
Hidden dimension	512
Attention heads	8 per layer
Feed-forward dimension	2,048
Acoustic input	80-bin log-Mel spectrogram
Audio sampling rate	16,000 Hz, mono
Audio window	Up to 30 seconds (3,000 Mel frames; 1,500 encoder positions)
FFT / hop length	400 / 160 samples
Decoder context	Up to 448 positions in the architecture
Tokenizer	Custom ByteLevel BPE trained from Common Voice Catalan and Spanish text
Vocabulary	24,000 BPE text tokens; 25,608 tokens total including special/reserved tokens
Training task	Language-conditioned transcription without timestamps

Training data and preparation

Data preparation followed the project's prepare.py and train.py scripts:

Read the official language-specific train.tsv, dev.tsv, and test.tsv manifests and resolve audio clip paths.
Normalize transcript text to Unicode NFC and normalize whitespace; reject invalid, missing, or empty examples.
Prioritize evaluation partitions when removing duplicate audio paths across the splits within a language.
Decode audio with FFmpeg, convert to mono 16 kHz floating-point PCM, and cap each training example at 30 seconds.
Extract the standard 80-bin Whisper log-Mel features.
Train the ByteLevel BPE tokenizer from scratch, using reservoir samples of up to 250,000 training sentences per language (vocab_size=24000, min_frequency=5, max_token_length=24). Add Whisper-compatible special tokens after the text vocabulary.

The training target for each clip is:

text

<|startoftranscript|> <|ca|> or <|es|> <|transcribe|> <|notimestamps|> TEXT <|endoftext|>

Training procedure

Table with columns: Phase, Updates in phase, Cumulative updates, Peak learning rate, Initial weights
Phase	Updates in phase	Cumulative updates	Peak learning rate	Initial weights
v2 — initial training	300,000	300,000	`2e-4`	Random initialization
v3 — continuation	50,000	350,000	`2e-5`	v2 (300K)
v4 — continuation	150,000	500,000	`1e-5`	v3 (350K)

Training parameters shared across the phases:

Table with columns: Hyperparameter, Value
Hyperparameter	Value
Framework	PyTorch + Hugging Face Transformers `Trainer`
Optimizer	AdamW (`adamw_torch`)
Scheduler	Cosine decay to zero; 5% warmup in each phase
Weight decay	`0.01`
Per-device training batch size	`1` in the reported single-GPU configuration
Gradient accumulation	`32` optimizer microbatches
Effective batch size	`32` examples per optimizer step (single GPU)
Max gradient norm	`1.0`
Dropout	`0.1`
SpecAugment	Enabled (`mask_time_prob=0.05`, `mask_time_length=10`)
Gradient checkpointing	Enabled
Random seed	`42`
Mixed precision	BF16 when supported by CUDA, otherwise FP16 on CUDA
Language sampling	80% Catalan / 20% Spanish throughout

The project used an NVIDIA RTX 5090 Laptop GPU (24 GB VRAM) for training, with a CUDA-enabled PyTorch environment. Exact throughput and training duration depend on hardware and data I/O.

Evaluation methodology

Common Voice: evaluated against the full dev.tsv and test.tsv partitions for each language of v27.0, rather than the 512-clip training-time validation subset.
FestCat and OpenSLR69: separately evaluated Catalan datasets used to assess out-of-domain performance. Their published Hugging Face dataset splits are labeled train; that label refers to the dataset packaging, not their use in training this model.
Language: supplied explicitly as ca or es, with transcription (transcribe) and timestamps disabled.
Decoding and inference: the reported Common Voice evaluator used batched Transformers generation with beam size 1 (greedy decoding), CUDA/FP16, and a fixed maximum decoder length.
WER/CER: computed with jiwer after Unicode NFC normalization, lowercasing, normalizing apostrophe variants and Catalan middle-dot spelling, removing most punctuation, and collapsing whitespace. Apostrophes, hyphens, accents, and middle dots are preserved where relevant.
RTF: measured from the evaluation implementation and batching configuration; it is not a hardware-independent benchmark.

The numbers above describe the 500K-step checkpoint, not necessarily other checkpoints or quantized variants.

Usage with Transformers

bash

pip install torch 'transformers>=4.50,<5' librosa

python

import torchimport librosafrom transformers import (    PreTrainedTokenizerFast,    WhisperFeatureExtractor,    WhisperForConditionalGeneration,) MODEL_ID = "YOUR_USERNAME/YOUR_MODEL_REPO"  # Replace with this model's Hub IDLANGUAGE = "ca"  # Or "es"AUDIO_FILE = "example.wav" assert LANGUAGE in {"ca", "es"}device = "cuda" if torch.cuda.is_available() else "cpu"dtype = torch.float16 if device == "cuda" else torch.float32 tokenizer = PreTrainedTokenizerFast.from_pretrained(MODEL_ID)model = WhisperForConditionalGeneration.from_pretrained(MODEL_ID)model = model.to(device=device, dtype=dtype).eval() # Use 80 Mel bins: do not load the frontend from Whisper large-v3 (128 Mel).extractor = WhisperFeatureExtractor(    feature_size=80,    sampling_rate=16000,    chunk_length=30,    n_fft=400,    hop_length=160,) # Explicit CA/ES language selection and no timestamps.model.generation_config.is_multilingual = Truemodel.generation_config.lang_to_id = {    f"<|{lang}|>": tokenizer.convert_tokens_to_ids(f"<|{lang}|>")    for lang in ("ca", "es")}model.generation_config.task_to_id = {    "transcribe": tokenizer.convert_tokens_to_ids("<|transcribe|>"),    "translate": tokenizer.convert_tokens_to_ids("<|translate|>"),}model.generation_config.no_timestamps_token_id = tokenizer.convert_tokens_to_ids("<|notimestamps|>")model.generation_config.forced_decoder_ids = None audio, _ = librosa.load(AUDIO_FILE, sr=16000, mono=True)if len(audio) > 30 * 16000:    raise ValueError("This example supports a maximum of 30 seconds of audio")input_features = extractor(    audio, sampling_rate=16000, return_tensors="pt").input_features.to(device=device, dtype=dtype) with torch.inference_mode():    output_ids = model.generate(        input_features=input_features,        language=LANGUAGE,        task="transcribe",        return_timestamps=False,        max_new_tokens=160,        do_sample=False,        use_model_defaults=False,  # Transformers 4.50+: avoid duplicate suppression processors    ) text = tokenizer.batch_decode(    output_ids, skip_special_tokens=True,    clean_up_tokenization_spaces=False,)[0]print(text.strip())

import torchimport librosafrom transformers import (    PreTrainedTokenizerFast,    WhisperFeatureExtractor,    WhisperForConditionalGeneration,) MODEL_ID = "YOUR_USERNAME/YOUR_MODEL_REPO"  # Replace with this model's Hub IDLANGUAGE = "ca"  # Or "es"AUDIO_FILE = "example.wav" assert LANGUAGE in {"ca", "es"}device = "cuda" if torch.cuda.is_available() else "cpu"dtype = torch.float16 if device == "cuda" else torch.float32 tokenizer = PreTrainedTokenizerFast.from_pretrained(MODEL_ID)model = WhisperForConditionalGeneration.from_pretrained(MODEL_ID)model = model.to(device=device, dtype=dtype).eval() # Use 80 Mel bins: do not load the frontend from Whisper large-v3 (128 Mel).extractor = WhisperFeatureExtractor(    feature_size=80,    sampling_rate=16000,    chunk_length=30,    n_fft=400,    hop_length=160,) # Explicit CA/ES language selection and no timestamps.model.generation_config.is_multilingual = Truemodel.generation_config.lang_to_id = {    f"<|{lang}|>": tokenizer.convert_tokens_to_ids(f"<|{lang}|>")    for lang in ("ca", "es")}model.generation_config.task_to_id = {    "transcribe": tokenizer.convert_tokens_to_ids("<|transcribe|>"),    "translate": tokenizer.convert_tokens_to_ids("<|translate|>"),}model.generation_config.no_timestamps_token_id = tokenizer.convert_tokens_to_ids("<|notimestamps|>")model.generation_config.forced_decoder_ids = None audio, _ = librosa.load(AUDIO_FILE, sr=16000, mono=True)if len(audio) > 30 * 16000:    raise ValueError("This example supports a maximum of 30 seconds of audio")input_features = extractor(    audio, sampling_rate=16000, return_tensors="pt").input_features.to(device=device, dtype=dtype) with torch.inference_mode():    output_ids = model.generate(        input_features=input_features,        language=LANGUAGE,        task="transcribe",        return_timestamps=False,        max_new_tokens=160,        do_sample=False,        use_model_defaults=False,  # Transformers 4.50+: avoid duplicate suppression processors    ) text = tokenizer.batch_decode(    output_ids, skip_special_tokens=True,    clean_up_tokenization_spaces=False,)[0]print(text.strip())

Recommended repository files

For a cleaner model distribution, upload the checkpoint weights and their matching tokenizer together with:

text

config.jsonmodel.safetensors               # or the actual sharded weight files

text

generation_config.jsontokenizer.jsontokenizer_config.jsonspecial_tokens_map.jsonpreprocessor_config.json        # recommended; missing in some v4 exportsREADME.md

generation_config.jsontokenizer.jsontokenizer_config.jsonspecial_tokens_map.jsonpreprocessor_config.json        # recommended; missing in some v4 exportsREADME.md

If your model export lacks preprocessor_config.json, generate it without changing the model's weights:

python

from transformers import WhisperFeatureExtractor WhisperFeatureExtractor(    feature_size=80,    sampling_rate=16000,    chunk_length=30,    n_fft=400,    hop_length=160,).save_pretrained("v4")  # Replace with the local model-export directory

from transformers import WhisperFeatureExtractor WhisperFeatureExtractor(    feature_size=80,    sampling_rate=16000,    chunk_length=30,    n_fft=400,    hop_length=160,).save_pretrained("v4")  # Replace with the local model-export directory

Intended use and limitations

Not established / not supported by training:

Reliable word- or segment-level timestamps (the timestamp tokens exist but were not trained).
Speech translation, speech synthesis, or recognition of the other reserved Whisper languages.
Robust automatic language identification, multilingual code-switching, or no-speech/VAD behavior without separate validation.
Reliable transcription of noisy conversations, overlapping voices, long recordings, or specialized terminology without task-specific tests.