GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ifs
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ifs
Model Tree
openai/whisper-large-v3
Input Modalities
Output Modalities
Supported Functionality
Error rates in percent on run 2's test set: 1,122 clips from 80 speakers, none of whom appear in this model's training data. CER and WER are computed after lowercasing and removing punctuation; raw CER compares the output exactly as produced.
| Model | CER | WER | raw CER |
|---|---|---|---|
| AigizK/wav2vec2-large-mms-1b-tatar, with its n-gram LM (CC BY-NC 4.0) | 3.2 | 19.3 | 10.6 |
| sammy786/wav2vec2-xlsr-tatar | 3.9 | 17.6 | 11.1 |
| this model | 4.3 | 18.2 | 6.0 |
| ifs/whisper-large-v3-tt-cv27-run1 | 5.2 | 22.8 | 6.7 |
| yasalma/whisper-finetuned-tt-asr | 7.9 | 32.3 | 9.7 |
| openai/whisper-large-v3, not fine-tuned | 32.1 | 89.3 | 33.5 |
It has the lowest raw CER of all models tested. Two wav2vec2 models have a lower normalized CER, but they write lowercase text without punctuation. The full table is in the write-up.
AigizK's language-model decoder never writes the letter й; it writes ⁇ instead, in 347 of the 1,122 clips. The table shows its output as it is; if every ⁇ is read as й, its CER drops to 2.2% and its WER to 10.3%.
This model has no score on the official Common Voice test split, because its training data includes most of that split's speakers (see below).
python
import torchfrom transformers import pipeline asr = pipeline( "automatic-speech-recognition", model="ifs/whisper-large-v3-tt-cv27-run2", dtype=torch.float16, # torch.float32 on a CPU device="cuda", # or "cpu")result = asr( "audio.mp3", return_timestamps=True, # needed for audio longer than 30 seconds generate_kwargs={ "language": "tatar", "task": "transcribe", "num_beams": 1, "no_repeat_ngram_size": 3, "repetition_penalty": 1.2, },)print(result["text"])The decoding settings are the ones used in evaluation; they follow BuzzASR. The weights are stored in float32 (6.2 GB); float16 halves the memory and works well for inference.
All clips come from the validated bucket of Common Voice Scripted Speech 27.0 Tatar. The official splits put 258 of 270 speakers in the test split and leave 2 for training, so this run re-split the data:
| Split | Clips | Speakers | How it was built |
|---|---|---|---|
| test | 1,122 | 80 | up to 40 clips each from 80 random speakers of the official test split, one clip per sentence |
| dev | 418 | 25 | the same, from 25 other speakers of the official test split |
| train | 26,855 | 165 | every other validated clip sharing no speaker and no sentence with dev or test |
One speaker accounts for 52% of the training clips. The exact splits are in
the splits/ folder of this repository. Labels keep their
original casing and punctuation.
Full fine-tuning following the BuzzASR recipe, with
finetune.py --resplit:
Hardware: one NVIDIA A100 80GB on RunPod. Software: PyTorch 2.8.0, Transformers 5.18.0 (full environment in the write-up).
MIT, like the base model. The training data is CC0.
import torchfrom transformers import pipeline asr = pipeline( "automatic-speech-recognition", model="ifs/whisper-large-v3-tt-cv27-run2", dtype=torch.float16, # torch.float32 on a CPU device="cuda", # or "cpu")result = asr( "audio.mp3", return_timestamps=True, # needed for audio longer than 30 seconds generate_kwargs={ "language": "tatar", "task": "transcribe", "num_beams": 1, "no_repeat_ngram_size": 3, "repetition_penalty": 1.2, },)print(result["text"])