navai-uz

whisper-tiny-uzbek

README

License: apache-2.0

Model overview

The NavAI Whisper family spans four sizes so you can trade accuracy for latency:

NavAI Uzbek Whisper family — macro-WER vs model size

Macro WER is the mean Word Error Rate across four held-out Uzbek test sets (see Evaluation). Lower is better. All four models live in the NavAI Whisper Collection.

Flagship — whisper-medium reaches 9.73 macro-WER / 8.81 micro-WER. See the full benchmark.

Benchmark results

The NavAI Whisper family on four held-out Uzbek test domains. Word Error Rate (%), lower is better.

Table with columns: model, params, macro, micro, feruza, FLEURS, USC, CV
model	params	macro	micro	feruza	FLEURS	USC	CV
`whisper-tiny`	39M	18.13	17.2	11.51	26.33	18.71	15.98
`whisper-base`	74M	15.13	14.05	10.38	22.4	14.95	12.79

WER %, lower is better. Macro = mean of the four domains (equal weight); micro = pooled by utterance.

Every result is re-decoded on one greedy harness and scored with uzbek_text_norm.

See how these compare across the open field in the open Uzbek Whisper leaderboard — every open Uzbek Whisper model, re-decoded on this same harness.

Getting started

python
# pip install "transformers>=4.40" librosa soundfile
import librosa
from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="navai-uz/whisper-tiny-uzbek",
    chunk_length_s=30,        # long-form audio is windowed automatically
)

# load & resample to 16 kHz — decodes wav/flac without ffmpeg
speech, sr = librosa.load("audio.wav", sr=16000)
result = asr(
    {"array": speech, "sampling_rate": sr},
    generate_kwargs={"language": "uz", "task": "transcribe"},
)
print(result["text"])

Passing a file path straight to asr("audio.wav", ...) also works, but the pipeline decodes it with ffmpeg, which must be installed separately (conda install -c conda-forge ffmpeg). Loading the array with librosa as above needs no ffmpeg.

Batched transcription:

python
files = ["a.wav", "b.wav", "c.wav"]
batch = [{"array": librosa.load(f, sr=16000)[0], "sampling_rate": 16000} for f in files]
for out in asr(batch, batch_size=8, generate_kwargs={"language": "uz"}):
    print(out["text"])

Best practices

Language token. Pass generate_kwargs={"language": "uz"} for best results.
Long audio. Set chunk_length_s=30; Whisper transcribes in 30-second windows.
Repetition guard. The shipped generation_config sets no_repeat_ngram_size=4, and all reported numbers are scored with it on.
Script. Output is Latin-script Uzbek (uzn_Latn).

Model data

Training dataset

753 hours of human-transcribed Uzbek speech, from four openly-licensed corpora:

Table with columns: Source, Version, License, Description
Source	Version	License	Description
Common Voice	Corpus 22.0 (`uz`)	CC0-1.0	crowd-sourced read speech (Mozilla Foundation)
FLEURS	`uz`	CC-BY-4.0	encyclopedic read speech (Google) — `train` split only; the disjoint `test` split is held out for

Explicitly excluded from training: machine pseudo-labels, scraped YouTube audio, and every evaluation test set (including FeruzaSpeech and FLEURS test). No evaluation data appears in training.

Attribution (USC, CC BY 4.0). "Our product uses ISSAI and TUIT Uzbek Speech Corpus, which is available under a Creative Commons Attribution 4.0 International License." — Musaev et al., USC: An Open-Source Uzbek Speech Corpus (arXiv:2107.14419).

Data preprocessing

Transcripts normalized with the released uzbek_text_norm package: Cyrillic→Latin folding, spelled-out numbers/ordinals, Latin apostrophe folding (oʻ/gʻ), and punctuation handling.
Audio resampled to 16 kHz mono; log-mel features padded to 30 seconds.

Training procedure

Full fine-tune of openai/whisper-tiny on the 753 h corpus above. Hyperparameters were selected per size by a hyperparameter search:

This size (search winner): HPO winner (lr 4.67e-5, wd 0.01, SpecAugment mtp 0.07)
Shared recipe: effective batch size 64 · AdamW · linear warmup (10%) → linear decay · up to 10 epochs · SpecAugment on the log-mel features · mixed precision (bf16 on A100 / fp16 on V100) · 4×GPU data-parallel (DDP) · repetition guard no_repeat_ngram_size=4.
Model selection: hyperparameters and the released checkpoint were selected on validation WER over a held-out USC split; the reported figures are then measured on the four-domain benchmark (FLEURS · Common Voice · USC · FeruzaSpeech).

Evaluation

Benchmark sets

Four held-out Uzbek test splits, spanning distinct acoustic and lexical conditions:

Table with columns: Set, Domain, Source
Set	Domain	Source
FLEURS	encyclopedic read speech	`google/fleurs` (`uz`, `test`)
Common Voice	crowd read speech	Common Voice Corpus 22.0 (`uz`, `test`)
USC	read speech

Scoring methodology

Decoding: greedy for every system — num_beams=1, no sampling. No model is advantaged by beam search.
Normalizer: the released uzbek_text_norm (v0.3.0, Apache-2.0), applied symmetrically to reference and hypothesis for every system: lowercase, all punctuation removed, hyphens → space (splitting hyphenated compounds, matching Whisper's BasicTextNormalizer), every apostrophe variant folded to the Uzbek okina (oʻ/gʻ), Cyrillic→Latin folding, and digits/ordinals spelled out. pip install uzbek-text-norm to reproduce every number in this card.
Uniform re-decode: every system — ours and baselines — is re-decoded on one harness with the repetition guard on (no_repeat_ngram_size=4), so none is advantaged by its own decode defaults.
Common Voice references were patched (2026-07-12) to remove a metadata bug that corrupted two references and inflated CV WER for every system. Any CV number published before that date — including in earlier revisions of this card — is unreliable.
Macro WER: mean of per-set WER (each domain weighted equally).

Limitations

Encyclopedic / named-entity speech is the weakest condition — rare foreign proper nouns are missed more often than common vocabulary.
Audio is processed in 30-second windows; extremely long files are chunked automatically.
Trained for Latin-script Uzbek; Cyrillic input is normalized at scoring but is not a primary target.
Far-field, noisy, or heavy-dialect audio is less well covered than clean read/conversational speech.

Ethical considerations

This model transcribes Uzbek speech and inherits biases present in its training data (speaker demographics, recording conditions, vocabulary). It should not be used as the sole basis for consequential decisions about individuals. Transcriptions of sensitive audio should be handled under appropriate privacy safeguards.

License

Released under the Apache License 2.0. Model weights are licensed independently of the training data; each training source retains its own license (documented above).

Citation

If you use these models or the uzbek_text_norm benchmark, please cite:

bibtex
@misc{navai_whisper_uzbek_2026,
  title        = {NavAI Whisper: Open Uzbek Speech Recognition},
  author       = {NavAI},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/navai-uz}}
}

Data sources

bibtex
@inproceedings{musaev2021usc,
  title     = {USC: An Open-Source Uzbek Speech Corpus and Initial Speech Recognition Experiments},
  author    = {Musaev, Muhammadjon and others},
  booktitle = {Speech and Computer (SPECOM)},
  year      = {2021},
  note      = {arXiv:2107.14419}
}
@misc{commonvoice22,
  title  = {Common Voice Corpus 22.0 (Uzbek)},
  author = {{Mozilla Foundation}},
  url    = {https://commonvoice.mozilla.org/en}
}
@article{conneau2022fleurs,
  title   = {FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech},
  author  = {Conneau, Alexis and others},
  journal = {arXiv:2205.12446},
  year    = {2022}
}

Datasets: Common Voice 22.0 · google/fleurs · issai/uzbek_speech_corpus · k2speech/FeruzaSpeech · UzbekVoice.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.