ASLP-lab

CN-MultiDialect-ASR

README

License: apache-2.0

Demo

Video demo with live waveforms and model transcriptions for Mandarin, English, four core dialects, and 15 ChinaVoices dialects.

Key Features

Mandarin–dialect balanced adaptation: improves Chinese dialect ASR while retaining Mandarin recognition.
Three-stage pipeline: CPT strengthens the Chinese ASR foundation, dialect SFT specializes for dialects, and OPSD refines the final checkpoint.
On-Policy Self-Distillation: trains on student-decoded prefixes with soft teacher targets, reducing the train–test mismatch of teacher-forced ASR training.
Drop-in inference: compatible with the official qwen-asr package.

Quickstart

Inference is compatible with Qwen3-ASR. We recommend installing the official qwen-asr package in a clean environment.

Environment Setup

bash
conda create -n qwen3-asr python=3.12 -y
conda activate qwen3-asr
pip install -U qwen-asr

For faster inference with the vLLM backend:

bash
pip install -U qwen-asr[vllm]

Model Download

You can load the model directly from Hugging Face, or download it locally first:

bash
# Hugging Face
pip install -U "huggingface_hub[cli]"
hf download ASLP-lab/CN-MultiDialect-ASR --local-dir ./CN-MultiDialect-ASR

# ModelScope (recommended for users in Mainland China)
pip install -U modelscope
modelscope download --model ASLP-lab/CN-MultiDialect-ASR --local_dir ./CN-MultiDialect-ASR

Python Inference

Load the model with Qwen3ASRModel.from_pretrained and call transcribe:

python
import torch
from qwen_asr import Qwen3ASRModel

model = Qwen3ASRModel.from_pretrained(
    "ASLP-lab/CN-MultiDialect-ASR",  # or "./CN-MultiDialect-ASR" for a local path
    dtype=torch.bfloat16,
    device_map="cuda:0",
    # attn_implementation="flash_attention_2",
    max_inference_batch_size=32,
    max_new_tokens=256,
)

results = model.transcribe(
    audio="path/to/audio.wav",
    language="Chinese",  # or None for automatic language detection
)

print(results[0].language)
print(results[0].text)

Batch inference is also supported:

python
results = model.transcribe(
    audio=[
        "path/to/mandarin.wav",
        "path/to/dialect.wav",
    ],
    language=["Chinese", "Chinese"],
)

for r in results:
    print(r.language, r.text)

For vLLM backend, streaming inference, and forced alignment, see the Qwen3-ASR repository.

Method Overview

Table with columns: Stage, Training data, Goal, Objective
Stage	Training data	Goal	Objective
`CPT`	Full Mandarin-dialect collection (`~100k` hours)	Build a stronger Chinese ASR foundation	Cross-entropy
`SFT`	Same sources with higher dialect sampling weight and a small Mandarin anchor	Lower dialect CER	Cross-entropy
`OPSD`	Dialect refinement subset (`~5k` hours)

At inference time, only the student pathway is used.

Performances

Dialect Overview

Public Dialect CER (%)

Internal Dialect CER (%)

Mandarin CER (%)

Citation

If you use this model, please cite:

bibtex
@misc{wang2026onpolicyselfdistillationmultidialectasr,
  title={On-Policy Self-Distillation for Multi-Dialect ASR: Mastering Dialects, Retaining Mandarin},
  author={Shuiyuan Wang and Bingshen Mu and Pengshen Zhang and Chengyou Wang and Yujie Liao and Chengdong Liang and Binbin Zhang and Qiangze Feng and Lei Xie},
  year={2026},
  eprint={2608.11898},
  archivePrefix={arXiv},
  primaryClass={eess.AS},
  url={https://arxiv.org/abs/2608.11898}
}

License

The released model is licensed under Apache 2.0.

Contact

For questions or collaborations, please contact wangshuiyuan@mail.nwpu.edu.cn.

You are also welcome to join our WeChat group for technical discussions and updates.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider