GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
LayerNorm
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
LayerNorm
Model Tree
Qwen/Qwen3-ASR-0.6B-hf
Input Modalities
Output Modalities
Supported Functionality
bash
# runtime deps for loading/transcribing (this model only needs transformers + torch + hqq)pip install transformers==5.15.1 torch==2.13.0 hqq==0.2.8.post1 # analysis dep (compare_baseline.py computes CER):pip install jiwerTranscribe a clip (recommended path — reconstructs the HQQ layers correctly):
python
import syssys.path.insert(0, "scripts")from infer_hqq_int4 import reconstruct_modelimport torchfrom transformers import AutoProcessor checkpoint_dir = "." # this repoprocessor = AutoProcessor.from_pretrained(checkpoint_dir)model = reconstruct_model(checkpoint_dir) # restores the HQQLinear layersmodel.eval() inputs = processor.apply_transcription_request( audio="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",).to(model.device, model.dtype) with torch.inference_mode(): out = model.generate(**inputs, max_new_tokens=256)gen = out[:, inputs["input_ids"].shape[1]:]print(processor.decode(gen, return_format="transcription_only")[0])Measured on an RTX 3060 12GB (CUDA 13.3, Ampere, compute 8.6) on a real Chinese corpus (AISHELL-1 short clips + a ~47 s long clip + an 8 dB-noise clip + the official demo clip, 10 samples total). CER = character error rate (CJK/latin alphanumerics only); latencies are the median of 3 generate calls.
| Metric | bf16 (baseline) | HQQ-INT4 | Δ |
|---|---|---|---|
| Checkpoint size (MB) | 1503.3 | 650.7 | −56.7% |
| GPU memory (MB) | 1492.5 | 643.1 | −56.9% |
| Mean CER vs. bf16 | — | 0.0063 | − |
| CER vs. AISHELL ground truth | 0.0231 | 0.0157 | − |
| Mean latency (s) | 0.90 | 1.76 | ×1.9 |
| Short-clip median latency (s) | 0.60 | 1.07 | ×1.8 |
| Long-clip (~47 s) latency (s) | 3.69 | 7.23 | ×2.0 |
Takeaways
Reproduce the numbers with scripts/compare_baseline.py
(needs the local Chinese test set described in scripts/compare_baseline.py).
bash
# 1) Quantize the base model to HQQ-INT4 (default output = this repo root)python scripts/quantize_hqq_int4.py --base Qwen/Qwen3-ASR-0.6B-hf # 2) Verify it reloads and transcribes correctlypython scripts/infer_hqq_int4.py --audio <some.wav> # 3) Compare against the bf16 baseline (Chinese test set)python scripts/compare_baseline.pybibtex
@article{Qwen3-ASR, title={Qwen3-ASR Technical Report}, author={Xian Shi, Xiong Wang, Zhifang Guo, Yongqi Wang, Pei Zhang, Xinyu Zhang, Zishan Guo, Hongkun Hao, Yu Xi, Baosong Yang, Jin Xu, Jingren Zhou, Junyang Lin}, journal={arXiv preprint arXiv:2601.21337}, year={2026}}# runtime deps for loading/transcribing (this model only needs transformers + torch + hqq)pip install transformers==5.15.1 torch==2.13.0 hqq==0.2.8.post1 # analysis dep (compare_baseline.py computes CER):pip install jiwerimport syssys.path.insert(0, "scripts")from infer_hqq_int4 import reconstruct_modelimport torchfrom transformers import AutoProcessor checkpoint_dir = "." # this repoprocessor = AutoProcessor.from_pretrained(checkpoint_dir)model = reconstruct_model(checkpoint_dir) # restores the HQQLinear layersmodel.eval() inputs = processor.apply_transcription_request( audio="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",).to(model.device, model.dtype) with torch.inference_mode(): out = model.generate(**inputs, max_new_tokens=256)gen = out[:, inputs["input_ids"].shape[1]:]print(processor.decode(gen, return_format="transcription_only")[0])# 1) Quantize the base model to HQQ-INT4 (default output = this repo root)python scripts/quantize_hqq_int4.py --base Qwen/Qwen3-ASR-0.6B-hf # 2) Verify it reloads and transcribes correctlypython scripts/infer_hqq_int4.py --audio <some.wav> # 3) Compare against the bf16 baseline (Chinese test set)python scripts/compare_baseline.py@article{Qwen3-ASR, title={Qwen3-ASR Technical Report}, author={Xian Shi, Xiong Wang, Zhifang Guo, Yongqi Wang, Pei Zhang, Xinyu Zhang, Zishan Guo, Hongkun Hao, Yu Xi, Baosong Yang, Jin Xu, Jingren Zhou, Junyang Lin}, journal={arXiv preprint arXiv:2601.21337}, year={2026}}