Intended / out-of-scope use
- Intended: transcribing Urdu telephone-call audio (customer-call
traffic, conversational telephony speech), decoded with
language=ur,
task=transcribe, greedy.
- Out of scope: general-purpose studio/broadcast Urdu (our internal
measurements place it behind general-purpose fine-tunes there), languages
other than Urdu, speaker identification, or any decision-making about
individuals.
Results
133 held-out SimpleRishta calls, zero caller overlap with training, corpus
C0 WER (normalized, PII-masked), identical decode settings for both models:
Table with columns: Model, WER on SR calls| Model | WER on SR calls |
|---|
openai/whisper-large-v3 (off-the-shelf) | 15.6% |
| this adapter | 8.9% |
Reference transcription: an independent high-accuracy Whisper-family model
(eval-only, never trained on). Both models scored identically against it.
Head-to-head on the same calls: this adapter wins 121 / loses 9 / ties 3.
Zero degenerate generations across all stability checks (repetition-rate and
length-ratio guards, clean greedy decoding). Aggregate protocol and numbers:
eval/aggregate-results.json.
Independent reproduction (live run): the 8.9% figure is re-derived
end-to-end on a clean GPU in this public Kaggle notebook — model loaded,
133 calls decoded, fresh WER printed against the certified value (observed
drift 0.0000):
r12-sr-eval-wer-proof.
Usage
import torch, soundfile as sffrom peft import PeftModelfrom transformers import WhisperForConditionalGeneration, WhisperProcessor base = "openai/whisper-large-v3-turbo"processor = WhisperProcessor.from_pretrained(base)model = WhisperForConditionalGeneration.from_pretrained(base)model = PeftModel.from_pretrained(model, "KhiredNetworks/PakUrdu-Conversational-ASR").merge_and_unload() speech, sr = sf.read("call.wav", dtype="float32")inputs = processor(speech, sampling_rate=sr, return_tensors="pt")ids = model.generate(inputs.input_features, language="ur", task="transcribe", do_sample=False)print(processor.batch_decode(ids, skip_special_tokens=True)[0])
A runnable version ships as inference_example.py.
Training data & governance
- Fine-tuned on real SimpleRishta customer-call audio (16 kHz telephone
Urdu; transcripts PII-masked before training; brand name preserved;
caller-group-safe train/dev split — no caller appears in both).
- Evaluation set frozen and never used in training.
- Training method: LoRA (PEFT), single epoch, fresh from the pinned base
revision — no continuation from any other checkpoint.
- No SimpleRishta audio, transcripts, or caller-identifying data are
published. Published results are aggregate-only.
- Full training/eval audit trail retained internally (sealed compute
records; certified numbers independently reproduced end-to-end on a
clean GPU run, drift 0.0000).
Transcription convention & inference settings
Urdu script output; evaluation text normalized under a corpus C0 protocol
(Unicode normalization, digit/punctuation folding, brand canonicalization,
PII masking) before scoring. Decode: greedy (do_sample=False),
language=ur, task=transcribe, 16 kHz mono input. FP16 on GPU / FP32 on
CPU both supported; results certified on a single T4.
Limitations
- Scores are vs a model reference, not human transcription.
- Evaluated on accepted (clean) call audio; performance on heavily noisy
or overlapped speech is not characterized here.
- Urdu only (
language=ur fixed in the intended decode config).
- Code/adapter: Apache-2.0 (see
LICENSE). Base model:
openai/whisper-large-v3-turbo — Whisper license and attribution chain
apply downstream.
- Citation: see
CITATION.cff (Khired Networks).
- Issues: use the repo's Issues tab.