zindagi-technologies

whisper-small-hindi

README

License: apache-2.0

Model details

Table

Base model	`openai/whisper-small` (244 M parameters; Radford et al., 2022)
Architecture	unchanged Whisper encoder–decoder; full-parameter fine-tune, no adapters
Language / task	Hindi (`hi`), transcription; no translation, no timestamps trained
Input	16 kHz mono audio, ≤ 30 s per segment (Whisper's fixed window); longer audio must be chunked by the caller
Output	Devanagari text in Kathbath's transcription style: no punctuation, numbers written as words, English loanwords in Devanagari; emitted no Latin or Arabic script and no digits on either test set
Weights	`model.safetensors`, 966,995,080 bytes (verified equal on both evaluation hosts and the Hub backup), SHA-256 `e78339cfa00a7d61592c1c82db6cbc783c8c0354e975cebe884add3efe148bed`
Licence (weights)	Apache-2.0 (inherited from the base model)

Training data provenance (source, version, preprocessing)

Training data source. AI4Bharat Kathbath, Hindi configuration, train split, obtained from the Hugging Face dataset ai4bharat/Kathbath (gated, auto-approved on accepting the terms; licence cc-by-4.0 per the dataset card's metadata, visible after access is granted; the packaging is CC0). Kathbath is described in Javed et al. (2022), IndicSUPERB (arXiv:2208.11761): 1,684 hours of read speech across 12 Indian languages from 1,218 contributors in 203 districts, collected with a maker–checker verification flow; every sentence is spoken once, and the corpus provides a speaker-disjoint ("test-unknown") test split. We verified on the data actually used that no test speaker appears in training (below).

Version. Dataset fingerprint of the train split as served on 2026-09-02: 9638f862dc22c910 (valid: 623dd9832d844bba), recorded in logs/download_manifest.json. Rows: 91,752 train / 3,151 valid. Measured duration: train 134.75 h (mean clip 5.29 s, max 20.06 s), valid 4.84 h (logs/train_data_hours.json). Hindi speakers in the training split: 101.

Preprocessing (scripts/train.py, configs/train_small.json):

audio decoded and resampled to 16 kHz mono; log-mel features (Whisper feature extractor, 80 mel bins, 3,000 frames) computed on the fly per batch, padded/truncated to 30 s;
clips kept if 0.3 s ≤ duration ≤ 30 s and the tokenized transcript is < 448 tokens (Whisper's decoder position limit) — no clip was removed by these filters (91,752 → 91,752, logs/train/train_full.log);
transcripts used verbatim as labels (no text normalization at training time); the Whisper tokenizer's language/task prefix set to <|hi|><|transcribe|><|notimestamps|>;
shuffling with seed 42; a fixed 500-utterance subset of valid (seed 42) used for dev-WER.

Training procedure

Table with columns: Hyper-parameter, Value
Hyper-parameter	Value
Epochs / optimizer steps	3 / 8,604
Effective batch size	32 (16 × 2 gradient accumulation)
Learning rate / schedule	1e-05 / linear decay, 500 warm-up steps
Optimizer	`adamw_bnb_8bit` (8-bit AdamW)
Precision	bf16; gradient checkpointing on
Seed	42
Model selection	best dev-WER checkpoint (evaluated every 1,000 steps) — the final step (12.62% at step 8,604)
Hardware / time

Dev-WER (Vistaar-normalized) at steps 1,000 … 8,000 and final: 22.19, 17.78, 15.03, 14.70, 14.19, 13.39, 13.20, 12.96, 12.62.

Evaluation

Identical protocol for the base and fine-tuned model (scripts/eval.py): greedy decoding, language="hindi", task="transcribe", no timestamps, max_new_tokens=440, bf16. Headline WER/CER use the AI4Bharat/Vistaar text normalization (strip punctuation and danda marks → indic-nlp-library Hindi normalizer → collapse whitespace), applied identically to reference and hypothesis; raw (un-normalized) figures are given alongside.

Table with columns: Test set, n, Model, WER, CER, WER (raw), CER (raw)
Test set	n	Model	WER	CER	WER (raw)	CER (raw)
Kathbath Hindi test (held-out)	1,927	whisper-small (zero-shot)	97.94% [90.00, 106.31]	58.64%	99.44%	59.25%
		fine-tuned (this work)	14.87% [14.17, 15.52]	5.18%	14.87%

Test-set integrity: the Kathbath test set is the official Vistaar benchmark archive (SHA-256 cb7d4fcdef4d3dc6…), cross-checked 100% against an independent mirror; a pre-training audit found 0 file-name overlaps and 2 verbatim transcript duplicates with the training split, which were excluded (n = 1,929 → 1,927); test and training speakers are disjoint (20 vs 101, overlap 0). FLEURS hi_in test: 418 recordings of 265 sentences from the official parquet (the dataset card's "920" is stale). The FLEURS runs are the corrected runs of 2026-09-03: a first version of the evaluation set had scored only 265 distinct recordings for the 418 rows because files were keyed on the FLEURS sentence id; the builder was fixed and both models re-evaluated (D2 report, §8).

Full report with per-utterance statistics, commands and the audit trail: D2 benchmark report (whitepaper/D2_BENCHMARK.md).

Intended use

Short-form Hindi speech-to-text (≤ 30 s segments; longer audio chunked by the caller) of read speech in the style of the training data (news-like sentences read aloud). Related uses such as announcements, dictation or voice form-filling prompts are plausible but were not evaluated.
As a starting point for further domain adaptation (customer calls, regional accents).
Research and evaluation of Hindi ASR fine-tuning under a fully documented protocol.

Limitations and out-of-scope use

Training data is read speech (134.75 h, 101 speakers). Conversational, telephone-quality, noisy or far-field audio was not in training and is not evaluated here; expect higher error.
Not evaluated on Hindi–English code-switching. Kathbath transcribes English loanwords in Devanagari; the model does the same and did not emit Latin script on either test set.
No long-form or timestamp capability was trained; the Whisper 30 s window applies. One FLEURS test clip (31.44 s) is truncated for both models.
Numerals are the weakest category. The model writes numbers as words even where a reference uses digits; on the 37 Kathbath test sentences containing digits its WER is 48.31% (FLEURS: 65 sentences, 28.03%). Downstream systems needing digits must post-process.
Speaker gender gap. Kathbath test WER is 11.51% on female-labelled and 18.34% on male-labelled recordings (n 971 / 956); no dialect or region breakdown is available.
Decoding. Evaluation used greedy decoding with a 440-token cap (an earlier 225-token cap truncated long FLEURS utterances and was superseded — D2 §8.2). Use max_new_tokens up to 440 for dense 30 s segments. The base model's near-100% zero-shot WER is dominated by repetition loops; with beam search (5) it reaches 59.81% on Kathbath, still four times the fine-tuned model's 14.87% (D2 §8.3).
The base model's zero-shot Hindi is poor (97.94% WER); the improvement figures should be read as adaptation of a general model to Hindi, not as a comparison with dedicated Hindi systems trained on 16–22× more data (see the whitepaper's positioning section).
Single training run, single seed; bootstrap 95% CIs: Kathbath 14.17–15.52, FLEURS 18.39–20.77 (WER %).

How to use

python
import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor

repo = "zindagi-technologies/whisper-small-hindi"
processor = WhisperProcessor.from_pretrained(repo, language="hindi", task="transcribe")
model = WhisperForConditionalGeneration.from_pretrained(repo, dtype=torch.float32).eval()

# `audio` is a 16 kHz mono float32 numpy array of up to 30 s
features = processor.feature_extractor([audio], sampling_rate=16000, return_tensors="pt").input_features
with torch.no_grad():
    ids = model.generate(input_features=features, max_new_tokens=440, num_beams=1)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

Reproducibility

Code, configuration, per-utterance evaluation outputs, training log and audit files are in the accompanying project repository (scripts/, configs/train_small.json, results/, logs/).
The fine-tuned Kathbath evaluation was re-run from a rebuilt evaluation set on 2026-09-03, on the same machine and environment, and reproduced bit-identically (0 of 1,927 hypotheses differ), and again on a second host with a different driver under protocol v3 (0 of 1,927 differ). Throughput (RTX 4090, greedy, batch 16, bf16, excluding model load): about 39× real time on the second host and about 160× on the first (identical output; hosts differ).
Exact package versions: logs/pod_env_freeze.txt (training host) and logs/pod/rerun/rerun_env_freeze.txt (protocol-v3 evaluation host); the scripts on both hosts were hash-verified against the committed versions (83088d3 and e0941aa).
Weights checksum manifest: logs/d1_model_manifest.json.

Licences and attribution

Weights: Apache-2.0, inherited from openai/whisper-small.
Training data: AI4Bharat Kathbath — CC-BY-4.0 (dataset card, ai4bharat/Kathbath; packaging CC0; the Hub access terms were accepted by the publisher on 2026-09-02). Attribution: AI4Bharat, IIT Madras — Javed et al., IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languages, arXiv:2208.11761 (2022).
Evaluation data: Kathbath Hindi test (as above, via the Vistaar benchmark archive, Bhogale et al., Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR, Interspeech 2023, arXiv:2305.15386); Google FLEURS — CC-BY-4.0 (dataset card, google/fleurs; Conneau et al., FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech, arXiv:2205.12446, 2022).
Base model: Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision, arXiv:2212.04356 (2022).
Text normalization follows the AI4Bharat Vistaar evaluation recipe (MIT-licensed repository AI4Bharat/vistaar) using indic-nlp-library.

Citation

bibtex
@misc{zindagi2026whispersmallhindi,
  title  = {whisper-small-hindi: openai/whisper-small fine-tuned for Hindi ASR on AI4Bharat Kathbath},
  author = {Zindagi Technologies Pvt Ltd},
  year   = {2026},
  note   = {Hugging Face model card, version 1.3}
}

Contact

Zindagi Technologies Pvt Ltd, New Delhi — https://zindagitech.com

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

zindagi-technologies

Model Tree

Base

openai/whisper-small

Fine-tuned

this model

Input Modalities

Audio

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer