inspiredclone101

qwen3-asr-indic-22lang-lora-v1

README

License: apache-2.0

Benchmark results (held-out, genuinely unseen test set)

Measured on 40 samples/language, sampled only from each language's own test*.jsonl split (never seen during training or validation — see the "Evaluation methodology" section below for why this matters). Forced-language decoding used on both sides (see Inference below) so the comparison isolates what fine-tuning changed.

Table with columns: Language, N, Base WER, Base CER, Adapter WER, Adapter CER
Language	N	Base WER	Base CER	Adapter WER	Adapter CER
Hindi	40	0.215	0.110	0.203	0.088
Urdu	40	0.928	0.760	0.191	0.090
Marathi	40	0.622	0.218	0.266	0.087
Sanskrit	40	0.857	0.270	0.345	0.078
Bengali	40	0.583	0.354	0.240	0.079
Gujarati	40	1.064	0.896	0.288	0.147
Sindhi	40	0.802	0.393	0.325	0.147
Punjabi	40	0.991	0.842	0.309	0.150
Nepali	38	0.850	0.358	0.348	0.145
Konkani	40	0.923	0.408	0.395	0.138
Maithili	40	0.767	0.332	0.360	0.133
Assamese	40	1.002	0.673	0.357	0.210
Dogri	40	0.823	0.405	0.395	0.183
Odia	40	1.085	0.933	0.415	0.178
Bodo	40	1.116	0.800	0.414	0.106
Kashmiri	40	0.999	0.811	0.471	0.238
Santali	40	1.050	0.886	0.464	0.237
Telugu	40	1.181	0.862	0.476	0.229
Kannada	40	1.254	0.908	0.551	0.280
Malayalam	40	1.118	0.719	0.543	0.244
Manipuri	40	1.205	1.010	0.556	0.361
Tamil	40	0.896	0.510	0.592	0.250

Average WER across all 22 languages: 0.924 (base) → 0.387 (this adapter) — a 58.2% relative reduction. Every language improves; best performers right now (WER < 0.3) are Hindi, Urdu, Marathi, Sanskrit, and Bengali. Weakest are Tamil, Manipuri, Malayalam, and Kannada — still meaningfully improved over base, but this run hasn't reached them as many times yet (single epoch, languages are interleaved rather than trained in sequential blocks).

Evaluation methodology

WER/CER were computed with jiwer after script/punctuation normalization. The "base" numbers are the stock Qwen/Qwen3-ASR-1.7B-hf model with no adapter attached, same decoding procedure. The test set is sampled exclusively from each language's test manifest split — rows this training run has never used for a gradient update or for validation/ checkpoint-selection — and additionally filtered to drop any row whose transcript's script doesn't match its labeled language (a labeling bug was found and fixed in one contributing dataset; see the training repo's README.md for details if you have access to it).

Architecture note

This is a single shared LoRA adapter trained across all 22 languages (rank 16, alpha 32, target_modules=["q_proj","v_proj"], dropout 0.05) — not one adapter per language. One frozen base model, one adapter, all languages mixed into one training stream.

Inference

Qwen3-ASR's built-in apply_transcription_request() helper only recognizes a ~30-language whitelist that doesn't cover most of the 22 languages this adapter targets. Force the language by directly prefilling the assistant turn instead — build the chat messages manually and set continue_final_message=True:

python

import torchimport librosafrom peft import PeftModelfrom transformers import Qwen3ASRForConditionalGeneration, Qwen3ASRProcessor BASE_MODEL = "Qwen/Qwen3-ASR-1.7B-hf"ADAPTER = "inspiredclone101/qwen3-asr-indic-22lang-lora-v1"DEVICE = "cuda"  # or "cpu" processor = Qwen3ASRProcessor.from_pretrained(BASE_MODEL)base_model = Qwen3ASRForConditionalGeneration.from_pretrained(    BASE_MODEL, torch_dtype=torch.bfloat16, attn_implementation="sdpa",).to(DEVICE)model = PeftModel.from_pretrained(base_model, ADAPTER)model.eval() audio, _ = librosa.load("your_audio.wav", sr=16000, mono=True)language = "Hindi"  # must match one of the 22 languages above, by name messages = [    {"role": "user", "content": [{"type": "audio", "audio": audio}]},    {"role": "assistant", "content": [{"type": "text", "text": f"language {language}<asr_text>"}]},]inputs = processor.apply_chat_template(    messages, tokenize=True, continue_final_message=True,    return_tensors="pt", return_dict=True,)inputs = {    k: (v.to(torch.bfloat16) if v.dtype == torch.float32 else v).to(DEVICE)    for k, v in inputs.items()} with torch.no_grad():    output_ids = model.generate(**inputs, max_new_tokens=200) decoded = processor.batch_decode(    output_ids[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0] # Output looks like "language Hindi<asr_text>वह यहाँ आया था" — keep only the part after <asr_text>.transcript = decoded.split("<asr_text>", 1)[-1].strip()print(transcript)

Why the manual prefill matters: <asr_text> is a fixed delimiter token in the model's own chat template — everything before it in the assistant turn is the forced language prompt we supply; everything the model generates after it is the actual transcription. Calling model.generate() without this prefill lets the model guess its own language, which it frequently gets wrong for lower-resource languages in this set (defaults to Hindi for many of them) — always force it this way.

Language names to use: exactly as listed at the top of this card (Hindi, Tamil, Telugu, Bengali, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Odia, Assamese, Urdu, Sanskrit, Nepali, Sindhi, Bodo, Dogri, Kashmiri, , , , ) — the string is inserted verbatim into the prompt, so it must match what the adapter was trained on.

Loading on CPU or with less GPU memory

Drop torch_dtype=torch.bfloat16 for torch.float32 on CPU, or use torch_dtype=torch.float16 with device_map="auto" if you have accelerate installed and limited VRAM. The base model is ~1.7B parameters (~3.5 GB in bf16); the adapter itself is ~19 MB.

Training data

~35,200 hours across 6 sources: a 15-dataset core corpus (IndicVoices, Shrutilipi, Kathbath, FLEURS, Rasa, and others), Vaani, a pseudo-labeled real-movie-audio dataset, Respin, Syspin, and open-slr. Coverage per language varies — some of the 22 languages (Bodo, Dogri, Sanskrit, Manipuri, Santali, Sindhi) only appear in the core corpus, while others draw from all 6 sources.

License

Apache 2.0, matching the base model. This adapter's weights are released as-is; it is an interim, non-final checkpoint from an active training run and comes with no performance guarantees.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

inspiredclone101

Model Tree

Base

Qwen/Qwen3-ASR-1.7B-hf

Adapter

this model

Input Modalities

Audio

Output Modalities

Text

Supported Functionality