Results (IndicTTS eval set, out-of-domain)
Table with columns: Voice, Baseline WER, LoRA WER, Relative Δ, Sarvam Saaras v3| Voice | Baseline WER | LoRA WER | Relative Δ | Sarvam Saaras v3 |
|---|
| Hindi Female (n=30) | 0.515 | 0.362 | −29.8% | 0.237 |
Closes 55% of the gap to Sarvam Saaras v3 (0.278 → 0.125 WER points).
Usage
from transformers import WhisperForConditionalGeneration, WhisperProcessor
from peft import PeftModel
processor = WhisperProcessor.from_pretrained("openai/whisper-small", language="Hindi", task="transcribe")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small")
model = PeftModel.from_pretrained(model, "manishehehe/whisper-small-hindi-lora")
model.eval()
ids = model.generate(input_features=input_features, language="hi", task="transcribe", max_new_tokens=225)
print(processor.tokenizer.batch_decode(ids, skip_special_tokens=True)[0])
Training
LoRA on q_proj/v_proj (r=32, α=64, dropout=0.05) — ~1.8M trainable params (0.7% of base). 500 steps, effective batch 32, LR 1e-3, fp16, single T4.
Code: https://github.com/manishehehe/whisper-indic-lora
Limitations
Small eval set (n=30); WER reductions are descriptive (no per-utterance significance test). FLEURS read speech differs in domain from IndicTTS studio recordings.