Junlaii

Bro-ASR-1.7B

Deploy Dedicated

README

License: apache-2.0

Intended use

Offline Japanese ASR for noisy conversational recordings.
Research and evaluation of device/noise-domain ASR adaptation.

It is not a safety-critical transcription system. Validate performance on your own audio domain before deployment.

Training

The model was produced by merging a LoRA adapter trained with the frozen lowrank_loraplus8_coverage recipe:

Base model: Qwen/Qwen3-ASR-1.7B-hf
Training pool: 3,000 Delivery_JP Japanese audio/text segments
LoRA: rank 8, alpha 16, dropout 0.1, applied to attention and MLP projections
LoRA+: base learning rate 8e-6, B-matrix ratio 8x
Sampling: shuffle without replacement; 500 micro-steps, gradient accumulation 2

The Delivery_JP audio and transcripts are not included in this repository.

Dataset and evaluation

Evaluation uses the 20,619-segment Delivery_JP Japanese manifest. The shared 3,000-segment training/development pool is excluded before scoring, leaving a strictly unseen evaluation set of 17,619 segments. Both models were run on the same ordered sample_id list with identical normalized ground truth; each side produced exactly 17,619 predictions with no duplicate IDs.

Decoding uses a Japanese language hint, beam size 5, and max_new_tokens=256. CER is character-level Levenshtein distance after the same whitespace normalization for both models. Macro-CER is the unweighted mean of per-segment CER.

Table with columns: Model, Evaluated segments, Micro-CER, Macro-CER, Character errors, Exact-match rate
Model	Evaluated segments	Micro-CER	Macro-CER	Character errors	Exact-match rate
Base Qwen3-ASR-1.7B	17,619	0.34295	0.41445	102,563	14.995%
Bro-ASR-1.7B	17,619	0.33763	0.39566	100,970	18.208%

Compared with the base model, Bro-ASR-1.7B reduces Micro-CER by 0.533 percentage points and character errors by 1,593. On paired per-segment comparison, Bro-ASR-1.7B is better / unchanged / worse on 5,681 / 7,775 / 4,163 segments, respectively.

Fixed-set audit score comparisons

The following four models were evaluated on the identical frozen 1,000 sample_id values with identical normalized ground truth and exact fractional audio time boundaries. This fixed-set audit is separate from the 17,619-segment strict-unseen evaluation above.

Table with columns: Fixed audit subset, Model, Segments, Micro-CER, Macro-CER, Character errors, Exact-match rate, Decoding
Fixed audit subset	Model	Segments	Micro-CER	Macro-CER	Character errors	Exact-match rate	Decoding
Delivery_JP frozen 1,000	Bro-ASR-1.7B	1,000	0.47119	0.49852	7,581	14.4%	Japanese, beam 5
Delivery_JP frozen 1,000	Qwen3-ASR-1.7B baseline	1,000

On this 1,000-segment audit, Bro-ASR-1.7B reduces Micro-CER by 1.454 percentage points and character errors by 234 relative to the paired Qwen baseline. The Cohere run used the official native transcription entry point with EOS handling; Qwen and Bro-ASR both used Japanese beam size 5.

The Whisper row is the untrained base model, not the historical Plan B fine-tuned checkpoint. It was reproduced with the frozen best_solution_snapshot.py TTA decoder and an asserted zero LoRA B-matrix. Its result is substantially weaker on this audit; the decode policy is reported separately because it uses three TTA speed factors while the Qwen runs use beam-5 decoding.

Quick start

Qwen3-ASR is not included in older stable Transformers releases. Install the current Transformers source first, as recommended by the upstream Qwen3-ASR model card:

bash
pip install "git+https://github.com/huggingface/transformers"
pip install torch soundfile

python
import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "Junlaii/Bro-ASR-1.7B"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
).eval()

inputs = processor.apply_transcription_request(
    audio="sample.wav", language="ja"
).to(model.device, model.dtype)
with torch.inference_mode():
    output = model.generate(**inputs, num_beams=5, max_new_tokens=256)
text = processor.decode(
    output[:, inputs["input_ids"].shape[1]:], return_format="transcription_only"
)[0]
print(text)

Limitations

The current adaptation is specifically motivated by Japanese noisy-recording conditions; it may regress on other languages or acoustic domains.
The full strict-unseen Delivery_JP evaluation has completed on 17,619 segments under the protocol described above.
Decoding settings affect results. The final comparison uses Japanese, beam size 5, and max_new_tokens=256.

License and attribution

Bro-ASR-1.7B follows the Apache-2.0 license indicated by its base checkpoint. Please cite and follow the upstream Qwen3-ASR model terms when using this model.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Junlaii

Model Tree

Base

Qwen/Qwen3-ASR-1.7B-hf

Fine-tuned

this model

Input Modalities

Audio

Output Modalities

Text

Supported Functionality