GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
waxal-benchmarking
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
waxal-benchmarking
Model Tree
openai/whisper-small
Input Modalities
Output Modalities
Supported Functionality
| Setting | WER | CER |
|---|---|---|
| Leave-Ewe-out (18 other languages, same recipe) | 101.4 | 56.8 |
| This model (Ewe only, same recipe) | 78.7 | 53.1 |
| Released per-language Whisper-Small (benchmark recipe) | 32.3 | n/a |
Test utterances scored: 1891.
All Ewe utterances in the WAXAL train split
(google/WaxalNLP), 16 kHz mono;
transcripts NFC-normalized, lower-cased, punctuation removed, diacritics preserved.
Labels longer than Whisper's 448-token limit are truncated.
| Hyperparameter | Value |
|---|---|
| Base | openai/whisper-small (244M) |
| Training languages | 1 (Ewe) |
| Steps | 4,000 |
| Optimizer | AdamW, lr 1e-5, 200 warmup steps |
| Batch size | 16 |
| Precision | fp16 |
| Language token | none (same as the leave-one-out model) |
| Hardware | 1x NVIDIA H200 |
Greedy decoding on the full Ewe WAXAL test split; WER and CER with jiwer on
NFC-normalized, lower-cased text with diacritics preserved.
python
from transformers import WhisperForConditionalGeneration, WhisperProcessorimport soundfile as sf model = WhisperForConditionalGeneration.from_pretrained("waxal-benchmarking/whisper-small-waxal-matched-ewe")processor = WhisperProcessor.from_pretrained("waxal-benchmarking/whisper-small-waxal-matched-ewe")wav, sr = sf.read("your_audio.wav") # 16 kHz monofeats = processor(wav, sampling_rate=16000, return_tensors="pt").input_featuresids = model.generate(feats)print(processor.batch_decode(ids, skip_special_tokens=True))Part of the WAXAL ASR Benchmark (arXiv:2606.02375).
bibtex
@article{waxalnet2026, title = {The WAXAL ASR Benchmark: Fine-Tuned Edge Models Across 19 African Languages}, author = {Olufemi, Victor Tolulope and Babatunde, Oreoluwa and Njema, Ramsey and others}, year = {2026}, note = {arXiv preprint arXiv:2606.02375}}Supported by Lynguallabs, Open Token, and CMU Africa.
from transformers import WhisperForConditionalGeneration, WhisperProcessorimport soundfile as sf model = WhisperForConditionalGeneration.from_pretrained("waxal-benchmarking/whisper-small-waxal-matched-ewe")processor = WhisperProcessor.from_pretrained("waxal-benchmarking/whisper-small-waxal-matched-ewe")wav, sr = sf.read("your_audio.wav") # 16 kHz monofeats = processor(wav, sampling_rate=16000, return_tensors="pt").input_featuresids = model.generate(feats)print(processor.batch_decode(ids, skip_special_tokens=True))@article{waxalnet2026, title = {The WAXAL ASR Benchmark: Fine-Tuned Edge Models Across 19 African Languages}, author = {Olufemi, Victor Tolulope and Babatunde, Oreoluwa and Njema, Ramsey and others}, year = {2026}, note = {arXiv preprint arXiv:2606.02375}}