Evaluation
The released source-alternating Whisper-small checkpoint achieves 85.47%
tolerance-two accuracy and 8.59% CER on the validation set, and 89.42%
tolerance-two accuracy and 8.13% CER on the task-provided local test set.
Table with columns: Evaluation surface, Items, tol2 accuracy (%), CER (%)| Evaluation surface | Items | tol2 accuracy (%) | CER (%) |
|---|
| Fixed validation set | 702 | 85.47 | 8.59 |
| Task-provided local test set | 1,900 | 89.42 | 8.13 |
| OOD set | 2,000 | 33.15 | 34.04 |
Metrics follow the task-defined character-level scoring protocol. The fixed
validation set controls checkpoint selection. The local-test and OOD sets
provide additional task-aligned and distribution-shift evaluation; they are
not used to choose checkpoints, language-model weights, or fusion methods.
Data provenance
The task-provided Cantonese materials were distributed for the preliminary ASR
task of the AI Dimsum
Cup, whose task page points to
the Cantonese Life Scenarios
Corpus.
The current dataset page is linked for provenance and is not asserted to be the
exact revision used during training. WenetSpeech-Yue supplies the broader
acoustic source. Training data is not included in this model repository.
Information identifying the exact training-time task-data snapshot is
available from the corresponding author by email; the contact address will be
added with the final author record.
Intended use
The checkpoint is intended for Cantonese automatic speech recognition research
and reproducible evaluation with the documented normalization and decoding
configuration. Users should evaluate it on their own acoustic domains and
transcription conventions before deployment.
The model card does not establish suitability for high-stakes, safety-critical,
or fully automated decision-making applications.
Offline inference
Download the complete repository because the bundled entry point loads model
assets with local_files_only=True:
hf download cantonese-asr-lab/whisper-small-cantonese-w500-adaptive \
--local-dir whisper-small-cantonese-w500-adaptive
python whisper-small-cantonese-w500-adaptive/predict.py \
--model_dir whisper-small-cantonese-w500-adaptive \
--audio_dir /path/to/audio \
--test_list /path/to/test.jsonl \
--output_jsonl predictions.jsonl
Input JSONL rows require an audio_path field. Output rows contain
audio_path and pred_text. Use --diagnostics-jsonl to save generated token
IDs alongside the prediction output.
The released defaults are language=zh, task=transcribe, num_beams=1, and
max_length=225. The bundled generation configuration also supplies
no_repeat_ngram_size=4 and repetition_penalty=1.05.
Reproducibility and source
- Technical Report: A Source-Alternating Curriculum for Cantonese Speech Recognition with Whisper
- Source repository: Vanxun-Hank/cantonese-asr
model.safetensors SHA-256:
a0f29a5a011213d5e4de34c40a02d42247255645f06e649e092d2dc495094370
- Flat release archive SHA-256:
b4fda8ac549d37d8cac9797950636d50ca28f74d8e5e76e2479c15de9b956bc5
config.json SHA-256:
53b4eb5c1c63510e9541417174df6ed709cd59e0b052492446ca1287088ee023
generation_config.json SHA-256:
3f2ced827b5a4b0241c4f2f8883cf13da2b00ece23c341d663333c9b07b8de64
A fixed 32-item offline audit found identical text and token outputs across the
raw checkpoint, extracted release package, and bundled inference path.
Model license
The released model weights are available under the
Apache License 2.0. Users remain
responsible for complying with the separate terms that apply to upstream
datasets, third-party models, and software dependencies.
Limitations
- The model was selected under one task-defined character protocol and should
be evaluated separately for other transcription standards.
- OOD performance is substantially lower than the two task-aligned surfaces;
domain-specific evaluation is necessary.
- Whisper uses the Chinese language token because its tokenizer does not expose
a dedicated Cantonese language token.
- Performance may vary with writing convention, acoustic domain, microphone,
noise, speaker population, code-switching, and utterance length.
- Training data is not redistributed here. Users must follow the upstream data
terms and applicable privacy requirements.
- The model repository does not itself contain the exact training-time
task-data snapshot.