Results
Table with columns: split, utterances, WER, CER| split | utterances | WER | CER |
|---|
| test (full, held-out) | 10,485 | 14.74% | 5.34% |
Scored on the full held-out test set, with the generation budget sized to
each clip's length. Training was monitored throughout on a small dev sample
and stopped at 72k of a planned 90k steps once that curve plateaued.
Training
- Base:
Qwen/Qwen3-ASR-0.6B-hf, frozen, bf16
- LoRA: r=64, alpha=64, dropout 0.05, on attention + MLP in both the
decoder and audio encoder — 59.2M trainable params (7.0% of 841.6M)
- Optimizer: batch 32 (2 × 16 grad accumulation), LR 1e-4 cosine, bf16,
gradient checkpointing
- Hardware: 1x NVIDIA RTX 5080
- Code: sayedshaun/qwen3-asr-finetune-bn
Data
753,378 Bengali utterances (1,174 h, 0.5–20s clips) from 9 sources:
Table with columns: source, utterances, hours, share| source | utterances | hours | share |
|---|
| IndicVoices (AI4Bharat) | 309,683 | 599.7 h | 51.1% |
| OpenSLR-53 — Large Bengali ASR training data set | 214,324 | 211.2 h | 18.0% |
| Vaani (ARTPARK / IISc) | 114,628 | 159.0 h |
IndicVoices and OpenSLR-53 alone make up 69% of the hours — worth keeping in
mind when judging what conditions this adapter is well calibrated for.
Usage
Requires transformers>=5.13.0 and peft>=0.21.
import torchfrom peft import PeftModelfrom transformers import AutoProcessor, Qwen3ASRForConditionalGeneration BASE = "Qwen/Qwen3-ASR-0.6B-hf"ADAPTER = "SayedShaun/Qwen3-ASR-0.6B-Bengali-LoRA" processor = AutoProcessor.from_pretrained(BASE)base_model = Qwen3ASRForConditionalGeneration.from_pretrained(BASE, dtype=torch.bfloat16)model = PeftModel.from_pretrained(base_model, ADAPTER).eval().to("cuda") conversation = [ {"role": "user", "content": [{"type": "audio", "path": "clip.wav"}]}, {"role": "assistant", "content": [{"type": "text", "text": "language Bengali<asr_text>"}]},]inputs = processor.apply_chat_template( conversation, tokenize=True, return_dict=True, padding=True, continue_final_message=True,).to(model.device, model.dtype) with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=256, do_sample=False) text = processor.batch_decode( out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]print(text.split("<asr_text>", 1)[-1].strip())
For clips longer than a few seconds, size max_new_tokens to the audio's
duration (~30 tokens/second is a safe rate) — a small fixed value will
silently truncate the transcript.
Preserving the original (multilingual) model
This is an unmerged LoRA adapter — the original multilingual weights are
untouched underneath it. For non-Bengali audio, disable the adapter on the
same loaded model instead of the Bengali language tag used above:
conversation = [ {"role": "user", "content": [{"type": "audio", "path": "clip.wav"}]}, {"role": "assistant", "content": [{"type": "text", "text": "language English<asr_text>"}]},]inputs = processor.apply_chat_template( conversation, tokenize=True, return_dict=True, padding=True, continue_final_message=True,).to(model.device, model.dtype) with model.disable_adapter(), torch.no_grad(): out = model.generate(**inputs, max_new_tokens=256, do_sample=False) text = processor.batch_decode( out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]print(text.split("<asr_text>", 1)[-1].strip())
model, processor, and the <asr_text> split above are the same ones
from Usage — this just wraps generation in disable_adapter() and swaps the
language tag for the language actually being spoken. Loading
Qwen3ASRForConditionalGeneration.from_pretrained(BASE, ...) on its own,
with no adapter involved at all, works the same way.
License
Released under the Apache License 2.0,
the same license as the base model
Qwen/Qwen3-ASR-0.6B-hf. Free
for commercial and non-commercial use, modification, and redistribution,
with attribution.