What's new in 1.1
- 🔀 Code-switch leap — ASCEND 10.59 → 9.60, CSZS 10.98 → 10.94; the English side of code-switch
(ASCEND-en) improves most.
- 🎯 Better on everything — CommonVoice, NTUML2021 lectures, and both code-switch sets all move down versus
TEA-ASR-1, on the same protocol.
- 🪶 Still a single drop-in checkpoint,
< 10 hours of public training audio, no runtime post-processing.
Key features
- 🎯 Built for Taiwan Mandarin — Traditional script and Taiwan-style word choice, produced by the model
itself.
- 🔀 Code-switch robust — handles natural zh-en mixing instead of translating Mandarin into English.
- 🧩 Drop-in Qwen3-ASR compatible — same loading and inference API as the base model; nothing else to install
or call.
- 🪶 Lightweight adaptation — a small decoder LoRA on a frozen audio encoder, trained on a few hours of public
audio, then merged for deployment.
Quick start
from qwen_asr import Qwen3ASRModel
model = Qwen3ASRModel.from_pretrained("JacobLinCool/TEA-ASR-1.1")
result = model.transcribe(audio="utterance.wav", language="Chinese")[0]
print(result.text)
Set language="Chinese" for Taiwan speech (recommended). You can also pass a context= string of hotwords
(names, jargon) for contextual biasing, exactly as with the base Qwen3-ASR.
Benchmark results
Mixed Error Rate (MER%, lower is better), all numbers from a single self-measured run under one protocol
(see Evaluation). Columns: the two TEA-ASR-1.1 models, the original (unadapted) Qwen3-ASR bases,
and two references — Breeze-ASR-25 (a Taiwan-specialist ASR) and Whisper-large-v3. Bold = this model.
Table with columns: Benchmark, TEA-ASR-1.1, TEA-ASR-1.1-mini, Qwen3-ASR-1.7B, Qwen3-ASR-0.6B, Breeze-ASR-25, Whisper-large-v3| Benchmark | TEA-ASR-1.1 | TEA-ASR-1.1-mini | Qwen3-ASR-1.7B | Qwen3-ASR-0.6B | Breeze-ASR-25 | Whisper-large-v3 |
|---|
| CommonVoice 19 (zh-TW) | 3.58 | 5.12 | 3.90 | 5.79 | 8.03 | 10.17 |
| ASCEND (zh-en) | 9.60 | 11.20 | 10.57 | 12.54 | 17.53 |
Generational improvement — TEA-ASR-1.1 vs TEA-ASR-1 (2B flagship, same protocol, lower is better):
Table with columns: Benchmark, TEA-ASR-1.1, TEA-ASR-1, Δ| Benchmark | TEA-ASR-1.1 | TEA-ASR-1 | Δ |
|---|
| CommonVoice 19 (zh-TW) | 3.58 | 3.64 | −0.06 |
| ASCEND (zh-en) | 9.60 | 10.59 | −0.99 |
| CSZS (zh-en) | 10.94 | 10.98 | −0.04 |
| NTUML2021 | 6.67 | 6.80 | −0.13 |
How to read this. TEA-ASR-1.1 is the flagship model on this page and posts the best error rate on every
benchmark in the suite — ahead of the Taiwan-specialist Breeze-ASR-25, far ahead of Whisper-large-v3, and below its
own predecessor TEA-ASR-1 on all four sets. TEA-ASR-1.1-mini delivers most of that quality at well under half
the parameters (780M vs 2B). The metric folds away script differences (see Evaluation), so it does not reflect
the decisive practical change: TEA-ASR emits Traditional script and Taiwan vocabulary natively, whereas the base
produces Simplified script.
Evaluation
- Metric — Mixed Error Rate (MER). Character Error Rate for Chinese and Word Error Rate for the English tokens,
computed jointly per utterance and micro-averaged.
- Content fold (applied uniformly to every dataset and every system). Before scoring, both the reference and
the hypothesis are normalized to a common form — converted to Simplified Chinese with OpenCC (
t2s),
lowercased, and stripped of punctuation. This isolates recognition from script style, so a Simplified-output
model and a Traditional-output model (TEA-ASR) are compared fairly on content. (TEA-ASR's actual output is
Traditional; the fold is only for scoring.)
- Decoding. TEA-ASR and Qwen3-ASR are decoded with
language=Chinese; Whisper-large-v3 and Breeze-ASR-25 use
their own automatic language detection. All systems are scored with the same code on the same public splits;
we do not import numbers reported elsewhere.
Table with columns: Dataset, What it tests, Eval split (n)| Dataset | What it tests | Eval split (n) |
|---|
| CommonVoice 19 (zh-TW) | Read Taiwan-Mandarin speech | full test (5013) |
| ASCEND | Spontaneous Mandarin–English code-switch conversation | full test (1315) |
| CSZS (zh-en) | Zero-resource code-switch benchmark | full test (3176) |
| NTUML2021 | Mandarin lecture speech (university ML course) | test[:2000] |
- No train/test leakage. Fine-tuning used only the training pools, disjoint from every evaluation split:
the NTUML2021 train split, the ASCEND train split, and a CommonVoice slice drawn from
validated_without_test (CommonVoice's official non-test pool, disjoint from its test split). Evaluation
therefore runs on the full, untouched CommonVoice / ASCEND / NTUML2021 test splits; CSZS is a separate
dataset not used in training at all. Every number above is leak-free.
How it was built
- Base
Qwen/Qwen3-ASR-1.7B (AuT audio encoder + Qwen3 decoder).
- Adaptation: a rank-16 decoder LoRA (plus a low-LR encoder LoRA) trained on under 10 hours of public
audio (CommonVoice zh-TW, ASCEND, NTUML2021, and TaiMECS),
with general + code-switch replay to preserve the base model's broad and bilingual ability. The 1.1 recipe
adds error-analysis-driven targeted supplements and English-preservation training so dense code-switch is
transcribed, not translated.
- Localization: Traditional-script + Taiwan-lexicon output is rendered through the model's own tokenizer
(the surface mapping is baked once at build time); there is no post-processing at inference — the Traditional
output comes straight from the model's own tokenizer decode.
- Packaging: the adapter is merged into the base and the localized tokenizer is shipped with it, so the
release is a single drop-in checkpoint that loads like stock Qwen3-ASR (decode verified bit-exact on 152k+
sequences).
- Decoding tip: pass
language="Chinese" for Taiwan speech; this also prevents translation-style outputs on
dense code-switch.
Limitations
- Scope: validated on the Qwen3-ASR family (0.6B and 1.7B); the released models load via the
qwen-asr package,
exactly like the base.
- Compact model trade-off: the 780M TEA-ASR-1.1-mini trails the 2B flagship on the hardest code-switch sets; for
heavy Mandarin–English mixing prefer TEA-ASR-1.1.
Acknowledgements
- Base model: Qwen3-ASR by Alibaba Cloud (Apache-2.0; underlying
weights remain subject to the Apache-2.0 license and its attribution/NOTICE terms).
- TaiMECS (CC-BY-4.0).
- Benchmarks: Common Voice (Mozilla), ASCEND (CAiRE), CSZS, NTU ML2021.
Citation
@misc{teaasr2026,
title = {Tokenizer-First Adaptation of Mandarin ASR to Taiwan Mandarin},
author = {TEA-ASR contributors},
year = {2026},
note = {TEA-ASR (Taiwan Everyday Audio); adapted from Qwen3-ASR}
}
The TEA-ASR adaptation and this checkpoint are released under the MIT License.
