Results (IndicTTS eval set, out-of-domain)
Table with columns: Voice, Baseline WER, LoRA WER, Relative Δ, Sarvam Saaras v3| Voice | Baseline WER | LoRA WER | Relative Δ | Sarvam Saaras v3 |
|---|
| Telugu Female | 1.062 | 0.704 | −33.8% | 0.363 |
| Telugu Male | 1.121 | 0.716 | −36.1% | 0.377 |
| Overall (n=60) | 1.091 | 0.709 | −35.0% | 0.370 |
Usage
from transformers import WhisperForConditionalGeneration, WhisperProcessor
from peft import PeftModel
processor = WhisperProcessor.from_pretrained("openai/whisper-small", language="Telugu", task="transcribe")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small")
model = PeftModel.from_pretrained(model, "manishehehe/whisper-small-telugu-lora")
model.eval()
ids = model.generate(input_features=input_features, language="te", task="transcribe", max_new_tokens=225)
print(processor.tokenizer.batch_decode(ids, skip_special_tokens=True)[0])
Training
LoRA on q_proj/v_proj (r=32, α=64, dropout=0.05) — ~1.8M trainable params (0.7% of base). 500 steps, effective batch 32, LR 1e-3, fp16, single T4.
Code: https://github.com/manishehehe/whisper-indic-lora
Limitations
Small eval set (n=60); WER reductions are descriptive (no per-utterance significance test). FLEURS read speech differs in domain from IndicTTS studio recordings.