Usage
!pip install -U "torchao>=0.16.0" peft
from transformers import WhisperProcessor, WhisperForConditionalGeneration
from peft import PeftModel
base = WhisperForConditionalGeneration.from_pretrained("openai/whisper-large-v3-turbo")
model = PeftModel.from_pretrained(base, "awaaz-se-alfaaz/whisper-turbo-v3-urdu")
processor = WhisperProcessor.from_pretrained("awaaz-se-alfaaz/whisper-turbo-v3-urdu")
Results
Benchmark WER (%) — LoRA fine-tuning
Table with columns: Dataset, Whisper-Turbo (Zero-Shot), Whisper-Turbo (Fine-Tuned), Relative Reduction| Dataset | Whisper-Turbo (Zero-Shot) | Whisper-Turbo (Fine-Tuned) | Relative Reduction |
|---|
| CSaLT | 27.51 | 20.77 | 24.5% |
| Common Voice v23 | 37.23 | 25.78 | 30.8% |
| FLEURS | 24.07 | 16.86 | 29.9% |
YouTube Evaluation Set (real-world, multi-speaker)
Table with columns: Model, WER (%)| Model | WER (%) |
|---|
| Whisper-Turbo | 25.33 |
| Whisper-Turbo Fine-Tuned | 24.20 |
SLM Post-Processing on YouTube Set
Table with columns: SLM Corrector, Whisper-Turbo, Whisper-Turbo Fine-Tuned| SLM Corrector | Whisper-Turbo | Whisper-Turbo Fine-Tuned |
|---|
| Gemma3-12B (4-bit) | 21.93 | 21.75 |
| Qwen3-14B (4-bit) | 23.58 | 22.39 |
| Tiny-Aya-Fire | 28.81 | 24.64 |
| Qwen3-4B | 30.57 | 29.41 |
Citation
[BibTeX once camera-ready is finalized]