Results
Held-out test, 500 utterances not seen during training or checkpoint selection, normalized CER
(whitespace/punctuation stripped), corpus-level (total edits / total reference characters):
Table with columns: Condition, Base model, This adapter| Condition | Base model | This adapter |
|---|
| Clean | 0.298 | 0.161 |
| Siren, 5dB SNR | 0.315 | 0.184 |
| Background noise, 10dB | 0.338 | 0.217 |
| Crowd babble, 10dB | 0.368 | 0.245 |
| Background noise, 0dB | 0.512 | 0.344 |
| Crowd babble, 5dB | 0.502 | 0.336 |
| Crowd babble, 0dB (noise = speech loudness) | 0.741 | 0.597 |
The clean-condition improvement is statistically significant (bootstrap 95% CI excludes 0). At 0dB crowd
babble the model is still frequently wrong — this remains an open problem, not solved by this adapter.
Given silence, background noise, or a siren alone (no speech), the base model hallucinates text
(e.g. "그렇죠.") 14–100% of the time depending on the noise type; this adapter outputs nothing in that
situation (0% across all three tested noise-only conditions, n=50 each).
Training
- Base:
Qwen/Qwen3-ASR-1.7B-hf, bf16
- LoRA: r=16, alpha=32, dropout=0.05, on decoder attention+MLP
(
q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj) — by suffix matching this also touches
the audio encoder's q/k/v projections (but not its out_proj/fc1/fc2, which use different names)
- Noise augmentation during training: per utterance, 47% chance of mixing synthetic noise
(crowd babble built from other training utterances, pink noise, or a siren tone) at 0–15dB SNR, 6%
chance of noise-only or silence input with an empty target (to suppress hallucination), 47% clean
- Optimizer: AdamW, lr 1e-4, cosine schedule with 3% warmup, effective batch 32 (4×2 grad-accum×4 GPU),
3 epochs, gradient checkpointing
- Checkpoint selection: corpus-level CER (not per-utterance average) on a periodic dev sample,
best of 3400 steps
- Hardware: 4× RTX 2080Ti (WSL2, DDP via
accelerate)
Data
20 hours from AI Hub's 119 emergency-call speech dataset
(Korean, citizen-to-119-dispatcher calls), balanced across 4 disaster categories at 5h each, utterances
with personal-information masking tags removed from training. Train/validation splits follow AI Hub's
own official split (verified: zero call-ID overlap between them). Noise is synthetic, not real recordings.
Known limitations (read before using in production)
- Domain mismatch with paramedic↔hospital calls. Training and evaluation are both citizen-to-119
report calls. The intended downstream use (ambulance crew calling a hospital) has not been tested at all.
- Noise is synthetic, generated to resemble crowd/background/siren noise — not verified against real
field recordings.
- Hotword/context prompting hurts, not helps. This adapter was trained without any
system prompt.
Passing a vocabulary hint via the system role at inference degrades CER (tested: 0.182→0.269 on a
related checkpoint, 95% CI excludes 0). Do not use this feature with this adapter without retraining.
- Outputs are frequently wrong in a way that reads as fluent, natural Korean — errors are not obviously
detectable from the text alone. Any deployment should keep a human in the loop to verify transcripts,
especially for safety-relevant details (symptoms, addresses, numbers).
- Short (<1s) and very noisy (0dB, same-loudness) utterances remain weak points.
Usage
import torchfrom peft import PeftModelfrom transformers import AutoProcessor, Qwen3ASRForConditionalGeneration BASE = "Qwen/Qwen3-ASR-1.7B-hf"ADAPTER = "Playedwell03/qwen3-asr-1.7b-119ko-v5" processor = AutoProcessor.from_pretrained(BASE)model = Qwen3ASRForConditionalGeneration.from_pretrained(BASE, dtype=torch.bfloat16, device_map="cuda")model = PeftModel.from_pretrained(model, ADAPTER)model.eval() inputs = processor.apply_transcription_request(audio="call.wav", language="Korean").to(model.device, model.dtype)output_ids = model.generate(**inputs, max_new_tokens=256)generated = output_ids[:, inputs["input_ids"].shape[1]:]print(processor.decode(generated, return_format="transcription_only")[0])
Requires transformers>=5.13.0 and peft.