kstorm77

whisper-small-danish

01 / USAGE

python

import torchfrom transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline model_id = "thorhojhus/whisper-small-danish"device = "cuda:0" if torch.cuda.is_available() else "cpu"dtype = torch.float16 if torch.cuda.is_available() else torch.float32 processor = AutoProcessor.from_pretrained(model_id)model = AutoModelForSpeechSeq2Seq.from_pretrained(    model_id, dtype=dtype, low_cpu_mem_usage=True,).to(device) transcribe = pipeline(    "automatic-speech-recognition",    model=model,    tokenizer=processor.tokenizer,    feature_extractor=processor.feature_extractor,    dtype=dtype,    device=device,)result = transcribe(    "audio.wav",    generate_kwargs={"language": "da", "task": "transcribe", "num_beams": 1},)print(result["text"])

02 / TRAINING

Training was started from stock Whisper Small. Filtered pseudo-labelled Danish speech was combined with the Danish portions of CoRal v3, Common Voice 26.0, FLEURS, FTSpeech and NST. Pseudo-labels were generated with whisper-large-v3-turbo-danish. The retained pool contained approximately 13,411 unique audio hours. Labels were filtered by confidence, no-speech probability, hallucination rules and a Danish/English vocabulary check.

Pseudo-labelled speech accounted for 100% of the sampling weight through step 50,000, 80% through step 140,000 and 70% thereafter. Human-labelled sources made up the remainder. Targets retained their casing, punctuation and numeral style; normalization was applied only for scoring.

The use of weak supervision and no-speech examples was inspired by the original Whisper paper. Waveform augmentation, SpecAugment and synthetic no-speech examples were included.

Table with columns: Setting, Value
Setting	Value
Steps	200,000
Optimizer	Muon for hidden matrices; AdamW for remaining parameters
Peak learning rate	Muon `5e-5`; AdamW `2.5e-5`
Weight decay	`0.01`
Schedule	1,000-step warmup, stable to 50,000, linear decay to 10% at 200,000
EMA	Per-step decay `0.9999`, applied every 50 steps
Precision	BF16 parameters with compensated updates; FP32 EMA
Maximum training segment	30 seconds

The following figures are cumulative audio sampled on the final training trajectory, including repeated passes. They are not unique dataset durations.

Table with columns: Source, Hours seen
Source	Hours seen
CoRal conversation	1,151.29
CoRal read-aloud	2,584.61
Common Voice Danish	139.94
FLEURS Danish	140.21
FTSpeech	4,182.40
NST Danish	1,938.15
Pseudo-labelled data	44,344.90
Total	54,481.50

Additional settings and provenance are recorded in training_recipe.json.

03 / EVALUATION

The checkpoint was evaluated using the public Danish ASR Leaderboard protocol with Transformers FP16 greedy Danish transcription. Standard decoding was used for short clips; timestamped sequential decoding was used above 30 seconds. Scoring included the leaderboard's Danish numeral normalization and did not remove filler words.

Table with columns: Test set, Samples, WER (%), CER (%)
Test set	Samples	WER (%)	CER (%)
CoRal conversation	8,438	24.01	15.02
CoRal read-aloud	9,122	14.16	5.73
Common Voice (`cv17_da`)	2,756	9.53	3.65
FLEURS `da_dk`	930

04 / LIMITATIONS

Performance outside Danish has not been validated after fine-tuning.
Long-form behavior was not directly trained and should be checked for the intended use.
Silence, noise and unfamiliar names can still produce incorrect text.

05 / LICENSE

The model weights are released under the MIT License. Training data are not redistributed and remain subject to their source terms. Attribution and the upstream Whisper notice are included in THIRD_PARTY_NOTICES.md.

06 / CREDITS

bibtex

@article{radford2022robust,  title={Robust Speech Recognition via Large-Scale Weak Supervision},  author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},  journal={arXiv preprint arXiv:2212.04356},  year={2022}}

whisper.cpp file ggml/ggml-whisper-small-danish-f16.bin: 487601984 bytes, sha256 f5cc3b03b53a60b1fce15a4e63abfd91da9723d67d2c2d42586f97ef9729d4e5, converted with whisper.cpp v1.9.1 (f049fff9) models/convert-h5-to-ggml.py from a float32 copy of model.safetensors.

whisper.cpp file ggml/ggml-whisper-small-danish-q8_0.bin: 264464624 bytes, sha256 55e73c83b66e38a34484c715325f602f3f5aba6d4db1958541d7244c032546ab, converted with whisper.cpp v1.9.1 (f049fff9) models/convert-h5-to-ggml.py from a float32 copy of model.safetensors and whisper-quantize … q8_0.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

kstorm77

Model Tree

Base

openai/whisper-small

Fine-tuned

this model

Input Modalities

Audio

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

01 / USAGE

python

import torchfrom transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline model_id = "thorhojhus/whisper-small-danish"device = "cuda:0" if torch.cuda.is_available() else "cpu"dtype = torch.float16 if torch.cuda.is_available() else torch.float32 processor = AutoProcessor.from_pretrained(model_id)model = AutoModelForSpeechSeq2Seq.from_pretrained(    model_id, dtype=dtype, low_cpu_mem_usage=True,).to(device) transcribe = pipeline(    "automatic-speech-recognition",    model=model,    tokenizer=processor.tokenizer,    feature_extractor=processor.feature_extractor,    dtype=dtype,    device=device,)result = transcribe(    "audio.wav",    generate_kwargs={"language": "da", "task": "transcribe", "num_beams": 1},)print(result["text"])

02 / TRAINING

The use of weak supervision and no-speech examples was inspired by the original Whisper paper. Waveform augmentation, SpecAugment and synthetic no-speech examples were included.

Table with columns: Setting, Value
Setting	Value
Steps	200,000
Optimizer	Muon for hidden matrices; AdamW for remaining parameters
Peak learning rate	Muon `5e-5`; AdamW `2.5e-5`
Weight decay	`0.01`
Schedule	1,000-step warmup, stable to 50,000, linear decay to 10% at 200,000
EMA	Per-step decay `0.9999`, applied every 50 steps
Precision	BF16 parameters with compensated updates; FP32 EMA
Maximum training segment	30 seconds

The following figures are cumulative audio sampled on the final training trajectory, including repeated passes. They are not unique dataset durations.

Table with columns: Source, Hours seen
Source	Hours seen
CoRal conversation	1,151.29
CoRal read-aloud	2,584.61
Common Voice Danish	139.94
FLEURS Danish	140.21
FTSpeech	4,182.40
NST Danish	1,938.15
Pseudo-labelled data	44,344.90
Total	54,481.50

Additional settings and provenance are recorded in training_recipe.json.

03 / EVALUATION

Table with columns: Test set, Samples, WER (%), CER (%)
Test set	Samples	WER (%)	CER (%)
CoRal conversation	8,438	24.01	15.02
CoRal read-aloud	9,122	14.16	5.73
Common Voice (`cv17_da`)	2,756	9.53	3.65
FLEURS `da_dk`	930

04 / LIMITATIONS

Performance outside Danish has not been validated after fine-tuning.
Long-form behavior was not directly trained and should be checked for the intended use.
Silence, noise and unfamiliar names can still produce incorrect text.

05 / LICENSE

06 / CREDITS

bibtex

@article{radford2022robust,  title={Robust Speech Recognition via Large-Scale Weak Supervision},  author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},  journal={arXiv preprint arXiv:2212.04356},  year={2022}}

whisper.cpp file ggml/ggml-whisper-small-danish-f16.bin: 487601984 bytes, sha256 f5cc3b03b53a60b1fce15a4e63abfd91da9723d67d2c2d42586f97ef9729d4e5, converted with whisper.cpp v1.9.1 (f049fff9) models/convert-h5-to-ggml.py from a float32 copy of model.safetensors.

whisper.cpp file ggml/ggml-whisper-small-danish-q8_0.bin: 264464624 bytes, sha256 55e73c83b66e38a34484c715325f602f3f5aba6d4db1958541d7244c032546ab, converted with whisper.cpp v1.9.1 (f049fff9) models/convert-h5-to-ggml.py from a float32 copy of model.safetensors and whisper-quantize … q8_0.

whisper-small-danish

README

01 / USAGE

02 / TRAINING

03 / EVALUATION

04 / LIMITATIONS

05 / LICENSE

06 / CREDITS

Explore FriendliAI today

README

01 / USAGE

02 / TRAINING

03 / EVALUATION

04 / LIMITATIONS

05 / LICENSE

06 / CREDITS