thorhojhus

whisper-large-v3-turbo-danish

Usage

python
import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline

model_id = "thorh/whisper-large-v3-turbo-danish"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    dtype=dtype,
    low_cpu_mem_usage=True,
).to(device)

transcribe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    dtype=dtype,
    device=device,
)

result = transcribe(
    "audio.wav",
    generate_kwargs={"language": "da", "task": "transcribe", "num_beams": 1},
)
print(result["text"])

Training

Training used the Danish portions of CoRal v3, Common Voice 26.0, FLEURS, FTSpeech and NST together with filtered pseudo-labelled data. The pseudo-labels were generated with openai/whisper-large-v3 and confidence-filtered before use.

Targets retained their original casing, punctuation and numeral style. Audio augmentation, SpecAugment and synthetic no-speech examples were included. The data design was inspired by the weak-supervision and no-speech principles in the original Whisper paper.

Table with columns: Setting, Value
Setting	Value
Steps	200,000
Optimizer	Muon for hidden matrices; AdamW for remaining parameters
Peak learning rate	Muon `5e-6`; AdamW `2.5e-6`
Weight decay	`0.01`
Schedule	1,000-step warmup, stable to 50,000, linear decay to 10% at 200,000
EMA	`0.9999` per step
Precision	BF16 parameters, FP8 hidden forward GEMMs, compensated FP32 optimizer updates
Maximum training segment	30 seconds
Training hardware	One NVIDIA GeForce RTX 4090
Active wall time	Approximately 56 hours

The following figures are cumulative audio sampled during training. They include repeated sampling and are not unique dataset durations.

Table with columns: Source, Hours seen
Source	Hours seen
CoRal conversation	485.45
CoRal read-aloud	894.45
Common Voice Danish	72.63
FLEURS Danish	74.06
FTSpeech	1,469.10
NST Danish	712.01
Pseudo-labelled data	9,200.04
Total	12,907.73

Additional recipe details are recorded in training_recipe.json.

Evaluation

The checkpoint was evaluated against the public Danish ASR Leaderboard protocol using greedy Danish transcription. Short clips used standard decoding; clips over 30 seconds used timestamped sequential long-form decoding. The leaderboard normalizer was applied, including Danish numeral normalization.

Table with columns: Test set, Samples, WER (%), CER (%)
Test set	Samples	WER (%)	CER (%)
CoRal conversation	8,438	20.05	11.92
CoRal read-aloud	9,122	12.93	4.80
Common Voice (`cv17_da`)	2,756	8.24	2.65
FLEURS `da_dk`	930

Exact revisions and decoding settings are recorded in training_recipe.json.

Limitations

Accuracy in languages other than Danish may be reduced.
Long-form behavior was not directly trained and should be validated separately.

License

The model weights are released under the MIT License. Training data are not redistributed and remain subject to their source terms. See THIRD_PARTY_NOTICES.md.

Citation

bibtex
@article{radford2022robust,
  title={Robust Speech Recognition via Large-Scale Weak Supervision},
  author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  journal={arXiv preprint arXiv:2212.04356},
  year={2022}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

thorhojhus

Model Tree

Base

openai/whisper-large-v3-turbo

Fine-tuned

this model

Input Modalities

Audio

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Usage

python
import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline

model_id = "thorh/whisper-large-v3-turbo-danish"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    dtype=dtype,
    low_cpu_mem_usage=True,
).to(device)

transcribe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    dtype=dtype,
    device=device,
)

result = transcribe(
    "audio.wav",
    generate_kwargs={"language": "da", "task": "transcribe", "num_beams": 1},
)
print(result["text"])

Training

Table with columns: Setting, Value
Setting	Value
Steps	200,000
Optimizer	Muon for hidden matrices; AdamW for remaining parameters
Peak learning rate	Muon `5e-6`; AdamW `2.5e-6`
Weight decay	`0.01`
Schedule	1,000-step warmup, stable to 50,000, linear decay to 10% at 200,000
EMA	`0.9999` per step
Precision	BF16 parameters, FP8 hidden forward GEMMs, compensated FP32 optimizer updates
Maximum training segment	30 seconds
Training hardware	One NVIDIA GeForce RTX 4090
Active wall time	Approximately 56 hours

The following figures are cumulative audio sampled during training. They include repeated sampling and are not unique dataset durations.

Table with columns: Source, Hours seen
Source	Hours seen
CoRal conversation	485.45
CoRal read-aloud	894.45
Common Voice Danish	72.63
FLEURS Danish	74.06
FTSpeech	1,469.10
NST Danish	712.01
Pseudo-labelled data	9,200.04
Total	12,907.73

Additional recipe details are recorded in training_recipe.json.

Evaluation

Table with columns: Test set, Samples, WER (%), CER (%)
Test set	Samples	WER (%)	CER (%)
CoRal conversation	8,438	20.05	11.92
CoRal read-aloud	9,122	12.93	4.80
Common Voice (`cv17_da`)	2,756	8.24	2.65
FLEURS `da_dk`	930

Exact revisions and decoding settings are recorded in training_recipe.json.

Limitations

Accuracy in languages other than Danish may be reduced.
Long-form behavior was not directly trained and should be validated separately.

License

The model weights are released under the MIT License. Training data are not redistributed and remain subject to their source terms. See THIRD_PARTY_NOTICES.md.

Citation

bibtex
@article{radford2022robust,
  title={Robust Speech Recognition via Large-Scale Weak Supervision},
  author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  journal={arXiv preprint arXiv:2212.04356},
  year={2022}
}

whisper-large-v3-turbo-danish

README

Usage

Training

Evaluation

Limitations

License

Citation

Explore FriendliAI today

README

Usage

Training

Evaluation

Limitations

License

Citation