wrice

whisper-tiny-grpo-5e2f49e-cv22-tiny-sftadaptive

Training data

Fine-tuned on cv22_index — all 78 Common Voice locales, streamed and decoded on the fly from the full training split. Each clip's language is pinned from its Common Voice locale during training.

Performance per language

Validation WER and CER at the best checkpoint, per Common Voice locale (also in the Evaluation Results metadata above):

Table with columns: Language, WER, CER
Language	WER	CER
`af`	0.867	0.360
`am`	1.000	0.814
`ar`	0.588	0.232
`as`	0.594	0.307
`az`	0.860	0.257
`ba`	0.844	0.359
`be`	0.815	0.277
`bg`	0.750	0.206
`bn`	0.469	0.242
`br`	0.829	0.376
`ca`	0.660	0.218
`cs`	0.656	0.130
`cy`	0.650	0.240
`da`	0.615	0.264
`de`	0.306	0.103
`el`	0.409	0.101
`en`	0.293	0.134
`es`	0.587	0.221
`et`	0.981	0.296
`eu`	0.708	0.184
`fa`	0.967	0.456
`fi`	0.577	0.153
`fr`	0.448	0.141
`gl`	0.480	0.124
`ha`	0.409	0.135
`he`	0.882	0.366
`hi`	0.321	0.154
`ht`	0.905	0.411
`hu`	1.056	0.746
`hy-AM`	1.000	0.480
`id`	0.846	0.486
`is`	0.935	0.419
`it`	0.553	0.153
`ja`	1.000	0.396
`ka`	1.000	0.822
`kk`	0.700	0.290
`ko`	0.522	0.244
`lo`	0.909	0.755
`lt`	1.000	0.368
`lv`	0.750	0.190
`mk`	0.727	0.194
`ml`	0.535	0.337
`mn`	0.853	0.330
`mr`	0.200	0.091
`mt`	0.812	0.254
`ne-NP`	0.480	0.241
`nl`	0.476	0.167
`nn-NO`	0.833	0.336
`oc`	0.720	0.270
`pa-IN`	0.400	0.195
`pl`	0.615	0.215
`ps`	0.833	0.529
`pt`	0.333	0.155
`ro`	0.167	0.046
`ru`	0.636	0.221
`sk`	0.826	0.281
`sl`	0.583	0.167
`sq`	0.963	0.450
`sr`	0.444	0.226
`sv-SE`	0.483	0.157
`sw`	0.767	0.214
`ta`	0.400	0.223
`te`	0.679	0.356
`tg`	0.800	0.270
`th`	0.655	0.248
`tk`	1.095	0.469
`tr`	0.750	0.276
`tt`	1.000	0.378
`uk`	0.625	0.205
`ur`	0.680	0.280
`uz`	0.818	0.260
`vi`	0.522	0.315
`yi`	0.810	0.439
`yo`	0.935	0.440
`yue`	1.000	0.275
`zh-CN`	1.000	0.271
`zh-HK`	1.000	0.321
`zh-TW`	0.667	0.250

How it was trained

Instead of cross-entropy against a single reference, for each audio clip the policy samples a group of num_generations transcriptions, scores each by a negated blend of word error rate, character error rate, and length / repetition penalties, and is nudged toward the better candidates with a clipped policy-gradient objective regularized by a per-token KL penalty to the frozen base model. Advantages are the group-relative, standardized rewards (A = (r - mean) / (std + eps)), so no value network is needed. The clip's language is pinned from its Common Voice locale, and the policy's own greedy transcriptions are scored as validation WER and CER.

Hyperparameters

Table with columns: Field, Value
Field	Value
Base model	`openai/whisper-tiny`
Dataset	`/data/cv22_index`
Learning rate	`1e-05`
Sampling temperature	`0.7`
Group size (generations/clip)	`8`
Reward weights	`{'cer': 1, 'wer': 1, 'length': 0.5, 'repetition': 0.5}`

Training curves

Pulled from the Weights & Biases run (static snapshot):

training curves

Usage

python

from transformers import pipeline asr = pipeline("automatic-speech-recognition", model="wrice/whisper-tiny-grpo-5e2f49e-cv22-tiny-sftadaptive")print(asr("audio.wav")["text"])

Limitations

A proof-of-concept GRPO recipe, not a tuned production system. WER and CER are reported on a held-out Common Voice validation slice after text normalization; real-world performance varies by domain, accent, language, and audio quality.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

wrice

Model Tree

Base

openai/whisper-tiny

Fine-tuned

this model

Input Modalities

Audio

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Training data

Fine-tuned on cv22_index — all 78 Common Voice locales, streamed and decoded on the fly from the full training split. Each clip's language is pinned from its Common Voice locale during training.

Performance per language

Validation WER and CER at the best checkpoint, per Common Voice locale (also in the Evaluation Results metadata above):

Table with columns: Language, WER, CER
Language	WER	CER
`af`	0.867	0.360
`am`	1.000	0.814
`ar`	0.588	0.232
`as`	0.594	0.307
`az`	0.860	0.257
`ba`	0.844	0.359
`be`	0.815	0.277
`bg`	0.750	0.206
`bn`	0.469	0.242
`br`	0.829	0.376
`ca`	0.660	0.218
`cs`	0.656	0.130
`cy`	0.650	0.240
`da`	0.615	0.264
`de`	0.306	0.103
`el`	0.409	0.101
`en`	0.293	0.134
`es`	0.587	0.221
`et`	0.981	0.296
`eu`	0.708	0.184
`fa`	0.967	0.456
`fi`	0.577	0.153
`fr`	0.448	0.141
`gl`	0.480	0.124
`ha`	0.409	0.135
`he`	0.882	0.366
`hi`	0.321	0.154
`ht`	0.905	0.411
`hu`	1.056	0.746
`hy-AM`	1.000	0.480
`id`	0.846	0.486
`is`	0.935	0.419
`it`	0.553	0.153
`ja`	1.000	0.396
`ka`	1.000	0.822
`kk`	0.700	0.290
`ko`	0.522	0.244
`lo`	0.909	0.755
`lt`	1.000	0.368
`lv`	0.750	0.190
`mk`	0.727	0.194
`ml`	0.535	0.337
`mn`	0.853	0.330
`mr`	0.200	0.091
`mt`	0.812	0.254
`ne-NP`	0.480	0.241
`nl`	0.476	0.167
`nn-NO`	0.833	0.336
`oc`	0.720	0.270
`pa-IN`	0.400	0.195
`pl`	0.615	0.215
`ps`	0.833	0.529
`pt`	0.333	0.155
`ro`	0.167	0.046
`ru`	0.636	0.221
`sk`	0.826	0.281
`sl`	0.583	0.167
`sq`	0.963	0.450
`sr`	0.444	0.226
`sv-SE`	0.483	0.157
`sw`	0.767	0.214
`ta`	0.400	0.223
`te`	0.679	0.356
`tg`	0.800	0.270
`th`	0.655	0.248
`tk`	1.095	0.469
`tr`	0.750	0.276
`tt`	1.000	0.378
`uk`	0.625	0.205
`ur`	0.680	0.280
`uz`	0.818	0.260
`vi`	0.522	0.315
`yi`	0.810	0.439
`yo`	0.935	0.440
`yue`	1.000	0.275
`zh-CN`	1.000	0.271
`zh-HK`	1.000	0.321
`zh-TW`	0.667	0.250

How it was trained

Hyperparameters

Table with columns: Field, Value
Field	Value
Base model	`openai/whisper-tiny`
Dataset	`/data/cv22_index`
Learning rate	`1e-05`
Sampling temperature	`0.7`
Group size (generations/clip)	`8`
Reward weights	`{'cer': 1, 'wer': 1, 'length': 0.5, 'repetition': 0.5}`

Training curves

Pulled from the Weights & Biases run (static snapshot):

training curves

Usage

python

from transformers import pipeline asr = pipeline("automatic-speech-recognition", model="wrice/whisper-tiny-grpo-5e2f49e-cv22-tiny-sftadaptive")print(asr("audio.wav")["text"])

whisper-tiny-grpo-5e2f49e-cv22-tiny-sftadaptive

README

Training data

Performance per language

How it was trained

Hyperparameters

Training curves

Usage

Limitations

Explore FriendliAI today

README

Training data

Performance per language

How it was trained

Hyperparameters

Training curves

Usage

Limitations