Benchmark results (held-out, genuinely unseen test set)
Measured on 40 samples/language, sampled only from each language's own test*.jsonl split (never seen
during training or validation — see the "Evaluation methodology" section below for why this matters).
Forced-language decoding used on both sides (see Inference below) so the comparison isolates what
fine-tuning changed.
Table with columns: Language, N, Base WER, Base CER, Adapter WER, Adapter CER| Language | N | Base WER | Base CER | Adapter WER | Adapter CER |
|---|
| Hindi | 40 | 0.215 | 0.110 | 0.203 | 0.088 |
| Urdu | 40 | 0.928 | 0.760 | 0.191 | 0.090 |
| Marathi | 40 | 0.622 | 0.218 | 0.266 | 0.087 |
| Sanskrit | 40 | 0.857 | 0.270 | 0.345 | 0.078 |
| Bengali | 40 | 0.583 | 0.354 | 0.240 | 0.079 |
| Gujarati | 40 | 1.064 | 0.896 | 0.288 | 0.147 |
| Sindhi | 40 | 0.802 | 0.393 | 0.325 | 0.147 |
| Punjabi | 40 | 0.991 | 0.842 | 0.309 | 0.150 |
| Nepali | 38 | 0.850 | 0.358 | 0.348 | 0.145 |
| Konkani | 40 | 0.923 | 0.408 | 0.395 | 0.138 |
| Maithili | 40 | 0.767 | 0.332 | 0.360 | 0.133 |
| Assamese | 40 | 1.002 | 0.673 | 0.357 | 0.210 |
| Dogri | 40 | 0.823 | 0.405 | 0.395 | 0.183 |
| Odia | 40 | 1.085 | 0.933 | 0.415 | 0.178 |
| Bodo | 40 | 1.116 | 0.800 | 0.414 | 0.106 |
| Kashmiri | 40 | 0.999 | 0.811 | 0.471 | 0.238 |
| Santali | 40 | 1.050 | 0.886 | 0.464 | 0.237 |
| Telugu | 40 | 1.181 | 0.862 | 0.476 | 0.229 |
| Kannada | 40 | 1.254 | 0.908 | 0.551 | 0.280 |
| Malayalam | 40 | 1.118 | 0.719 | 0.543 | 0.244 |
| Manipuri | 40 | 1.205 | 1.010 | 0.556 | 0.361 |
| Tamil | 40 | 0.896 | 0.510 | 0.592 | 0.250 |
Average WER across all 22 languages: 0.924 (base) → 0.387 (this adapter) — a 58.2% relative
reduction. Every language improves; best performers right now (WER < 0.3) are Hindi, Urdu, Marathi,
Sanskrit, and Bengali. Weakest are Tamil, Manipuri, Malayalam, and Kannada — still meaningfully
improved over base, but this run hasn't reached them as many times yet (single epoch, languages are
interleaved rather than trained in sequential blocks).
Evaluation methodology
WER/CER were computed with jiwer after script/punctuation
normalization. The "base" numbers are the stock Qwen/Qwen3-ASR-1.7B-hf model with no adapter
attached, same decoding procedure. The test set is sampled exclusively from each language's test
manifest split — rows this training run has never used for a gradient update or for validation/
checkpoint-selection — and additionally filtered to drop any row whose transcript's script doesn't
match its labeled language (a labeling bug was found and fixed in one contributing dataset; see the
training repo's README.md for details if you have access to it).
Architecture note
This is a single shared LoRA adapter trained across all 22 languages (rank 16, alpha 32,
target_modules=["q_proj","v_proj"], dropout 0.05) — not one adapter per language. One frozen base
model, one adapter, all languages mixed into one training stream.
Inference
Qwen3-ASR's built-in apply_transcription_request() helper only recognizes a ~30-language whitelist
that doesn't cover most of the 22 languages this adapter targets. Force the language by directly
prefilling the assistant turn instead — build the chat messages manually and set
continue_final_message=True:
import torchimport librosafrom peft import PeftModelfrom transformers import Qwen3ASRForConditionalGeneration, Qwen3ASRProcessor BASE_MODEL = "Qwen/Qwen3-ASR-1.7B-hf"ADAPTER = "inspiredclone101/qwen3-asr-indic-22lang-lora-v1"DEVICE = "cuda" # or "cpu" processor = Qwen3ASRProcessor.from_pretrained(BASE_MODEL)base_model = Qwen3ASRForConditionalGeneration.from_pretrained( BASE_MODEL, torch_dtype=torch.bfloat16, attn_implementation="sdpa",).to(DEVICE)model = PeftModel.from_pretrained(base_model, ADAPTER)model.eval() audio, _ = librosa.load("your_audio.wav", sr=16000, mono=True)language = "Hindi" # must match one of the 22 languages above, by name messages = [ {"role": "user", "content": [{"type": "audio", "audio": audio}]}, {"role": "assistant", "content": [{"type": "text", "text": f"language {language}<asr_text>"}]},]inputs = processor.apply_chat_template( messages, tokenize=True, continue_final_message=True, return_tensors="pt", return_dict=True,)inputs = { k: (v.to(torch.bfloat16) if v.dtype == torch.float32 else v).to(DEVICE) for k, v in inputs.items()} with torch.no_grad(): output_ids = model.generate(**inputs, max_new_tokens=200) decoded = processor.batch_decode( output_ids[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0] # Output looks like "language Hindi<asr_text>वह यहाँ आया था" — keep only the part after <asr_text>.transcript = decoded.split("<asr_text>", 1)[-1].strip()print(transcript)
Why the manual prefill matters: <asr_text> is a fixed delimiter token in the model's own chat
template — everything before it in the assistant turn is the forced language prompt we supply;
everything the model generates after it is the actual transcription. Calling model.generate()
without this prefill lets the model guess its own language, which it frequently gets wrong for
lower-resource languages in this set (defaults to Hindi for many of them) — always force it this way.
Language names to use: exactly as listed at the top of this card (Hindi, Tamil, Telugu,
Bengali, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Odia, Assamese, Urdu,
Sanskrit, Nepali, Sindhi, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri,
Santali) — the string is inserted verbatim into the prompt, so it must match what the adapter was
trained on.
Loading on CPU or with less GPU memory
Drop torch_dtype=torch.bfloat16 for torch.float32 on CPU, or use torch_dtype=torch.float16 with
device_map="auto" if you have accelerate installed and limited VRAM. The base model is ~1.7B
parameters (~3.5 GB in bf16); the adapter itself is ~19 MB.
Training data
~35,200 hours across 6 sources: a 15-dataset core corpus (IndicVoices, Shrutilipi, Kathbath, FLEURS,
Rasa, and others), Vaani, a pseudo-labeled real-movie-audio dataset, Respin, Syspin, and open-slr.
Coverage per language varies — some of the 22 languages (Bodo, Dogri, Sanskrit, Manipuri, Santali,
Sindhi) only appear in the core corpus, while others draw from all 6 sources.
License
Apache 2.0, matching the base model. This adapter's weights are released as-is; it is an interim,
non-final checkpoint from an active training run and comes with no performance guarantees.