askhaYerga

whisper-turbo-ksc2-kk-spontaneous-lora

Intended use

Verbatim transcription, where hesitations and repetitions are part of what must be recorded: call-centre and hotline recordings, forensic, clinical and linguistic transcription, conversation analysis, disfluency research.

Not recommended for

Content-oriented downstream tasks. On a named-entity proxy, with filled pauses removed from reference and hypothesis alike, this adapter is not significantly better than the base model (F1 0.725 vs 0.705, p = 0.33). If you need entities, topics or intents rather than a verbatim transcript, the base model is enough.
Prepared or read speech. No benefit is expected; none is measured (−0.23 p.p., p = 0.25).
Languages other than Kazakh, including Kazakh–Russian code-switched speech beyond what KSC2 contains.

Usage

python

import torch, soundfile as sffrom transformers import WhisperForConditionalGeneration, WhisperProcessorfrom peft import PeftModel BASE = "abilmansplus/whisper-turbo-ksc2"ADAPTER = "askhaYerga/whisper-turbo-ksc2-kk-spontaneous-lora" processor = WhisperProcessor.from_pretrained(BASE)model = WhisperForConditionalGeneration.from_pretrained(BASE, torch_dtype=torch.float16)model = PeftModel.from_pretrained(model, ADAPTER)model = model.merge_and_unload().to("cuda").eval()   # merging is optional wav, sr = sf.read("utterance.wav")                   # 16 kHz monoinputs = processor(wav, sampling_rate=16000, return_tensors="pt")feats = inputs.input_features.to("cuda", dtype=torch.float16) # input_features passed by keyword: PeftModelForSeq2SeqLM.generate() takes no# positional arguments, so this call works with or without merge_and_unload()ids = model.generate(input_features=feats, num_beams=5, max_new_tokens=220,                     language="kazakh", task="transcribe")print(processor.batch_decode(ids, skip_special_tokens=True)[0])

Training data

7,139 utterances (16.39 h) of the Train partition of the Kazakh Speech Corpus 2, selected as spontaneous by a deterministic text-based spontaneity filter and referred to as the Golden Dataset. Selection uses weighted, length-normalised counts of lexical traces of online planning — lengthened hesitations and backchannels, isolated single-letter vocalisations, discourse markers, repair phrases, non-overlapping repetitions outside a stop-list of lexicalised Kazakh reduplication, and Russian function words and morphologically blended forms.

KSC2 audio and transcripts are not redistributed here. The utterance-level manifest with the register labels, the filter and the lexicons are released separately (10.5281/zenodo.23357132); the audio must be obtained from ISSAI under the KSC2 licence.

Training configuration

Table with columns: Hyperparameter, Value
Hyperparameter	Value
Rank r / scaling α	16 / 32
Target modules	query and value projections in self- and cross-attention, encoder and decoder
Trainable parameters	3.28 M — 0.41 % of the base model
Dropout	0.05
Learning rate / schedule	1e-4, cosine, 10 % warm-up
Effective batch size	32 (8 × 4 gradient accumulation)
Epochs	3, fixed; no checkpoint selection
Augmentation	SpecAugment, on the fly; no synthetic disfluency injection
Precision	fp16
Hardware / time	NVIDIA RTX 4090 (24 GB); ≈ 35 min per run
Adapter size	≈ 13 MB

Epochs were fixed in advance and no checkpoint selection was performed: the only tuning data available was drawn from the partition the base model had already seen, and such a set cannot discriminate between checkpoints.

Evaluation

Sets. 584 spontaneous utterances (1.52 h) from 24 recording-session groups, and a prepared control of 2,000 utterances drawn from the same 24 sessions, so that the comparison holds the recording channel and, approximately, the speaker population fixed. All sets are disjoint from the training data at the level of recording sessions.

Scoring conditions. Identical text normalisation of references and hypotheses, then (a) filled pauses retained, (b) filled pauses removed from both sides, (c) canonical — every filler token, whatever its spelling, mapped to one symbol. Condition (c) is the primary one: the vowel of a Kazakh filled pause is written inconsistently in the references, and a model should not be penalised for a spelling on which human annotators disagree.

Statistics. Paired bootstrap with 10,000 resamples of recording sessions, not of utterances. Decoding: beam 5, forced Kazakh language token, no external language model.

Word error rate (%)

Table with columns: Base — spont., Adapter — spont., Base — prep., Adapter — prep.
	Base — spont.	Adapter — spont.	Base — prep.	Adapter — prep.
(c) canonical	13.48	10.78 ± 0.04	12.22	11.99 ± 0.05
(a) fillers retained	17.78	15.42 ± 0.06	12.67	12.82 ± 0.05
(b) fillers removed	11.27	11.22 ± 0.04	11.67	11.47 ± 0.03

Canonical WER on spontaneous speech: −2.70 p.p., 95 % CI [−4.75; −1.22], p < 0.001. On prepared speech: −0.23 p.p., 95 % CI [−0.66; +0.14], p = 0.25 — inside a ±1.0 p.p. tolerance fixed before the experiment.

Character error rate (%)

Table with columns: Base — spont., Adapter — spont., Base — prep., Adapter — prep.
	Base — spont.	Adapter — spont.	Base — prep.	Adapter — prep.
(c) canonical	6.63	4.89 ± 0.07	4.89	4.77 ± 0.02
(a) fillers retained	7.83	6.53 ± 0.06	4.84	4.92 ± 0.02
(b) fillers removed	4.72	4.59 ± 0.07	4.53	4.43 ± 0.01

What happens to the filled pauses (%)

Table with columns: Model, Transcribed exactly, Other spelling, Omitted, Replaced by a word
Model	Transcribed exactly	Other spelling	Omitted	Replaced by a word
Base	45.6	28.7	23.0	2.7
Adapter	63.1	30.6	4.6	1.8

The share rendered in another spelling does not fall. The adapter did not learn the transcribers' orthography of hesitations — it learned to stop deleting them. Exact-match accuracy rises only because omitted pauses are now emitted. This is why canonical scoring, not plain WER, is the metric to compare on.

Reproducibility

Three adapters were trained with seeds 42, 43 and 44; this repository contains one of them, and the figures above are means over the three runs with the seed-to-seed standard deviation shown. Evaluating this checkpoint alone will therefore give a value within that spread rather than the mean exactly.

The evaluation scripts, the spontaneity filter, the versioned lexicons and the fixed identifier lists of every evaluation set are released at 10.5281/zenodo.23357132 and github.com/AskhaYergali/kazakh-spontaneous-speech.

Limitations

Filled-pause spelling is not a reliable category in Kazakh. In a blind listening audit of 30 reference/hypothesis disagreements the listener sided with the model 15 times and with the reference 15 times. Standard WER penalises a disagreement that humans share, which is why canonical scoring is used here.
Register-targeted data is not the active ingredient. A size- and duration-matched random sample of KSC2 Train recovers 93 % of the aggregate gain, and an adapter trained on prepared speech from the same recording sessions reaches the same canonical WER. What curated spontaneous data buys is a lower residual omission rate (4.6 % vs 6.2 %, p < 0.001), not a lower word error rate.
No speaker metadata. KSC2 provides none; recording sessions serve as an approximate speaker proxy, so speaker disjointness between training and evaluation is substantially reduced but not strictly guaranteed.
Third-party base model. The training normalisation of whisper-turbo-ksc2 is not documented, so the size of the fluency prior characterises that model rather than Whisper as such.
Material. Broadcast, podcast, radio, talk-show and parliamentary speech; utterances of 2.5–22 s. Behaviour on telephone-band audio, far-field recordings or long-form input is untested.
Offline transcription only. Streaming with bounded latency was not evaluated.

Licence and attribution

Released under the MIT licence, matching the base model.

This adapter is a derivative of abilmansplus/whisper-turbo-ksc2 (MIT), itself derived from openai/whisper-large-v3-turbo (MIT), and was trained on the Kazakh Speech Corpus 2, released by ISSAI under CC BY 4.0. Attribution for the corpus:

Mussakhojayeva, S.; Khassanov, Y.; Varol, H.A. KSC2: An Industrial-Scale Open-Source Kazakh Speech Corpus. Proceedings of Interspeech 2022, Incheon, 2022, pp. 1367–1371. https://doi.org/10.21437/Interspeech.2022-421

Citation

bibtex

@article{yergaliyev2023spontaneous,  title   = {Spontaneous Speech and Its Features Are Taken into Account             When Creating Recognition Programs},  author  = {Yergaliyev, Askhat and Sharipbay, Altynbek and Baibulekova, Lyailya},  journal = {Ing\'enierie des Syst\`emes d'Information},  volume  = {28},  number  = {2},  pages   = {443--450},  year    = {2023},  doi     = {10.18280/isi.280220}} @dataset{yergaliyev2026manifest,  title     = {Kazakh Spontaneous Speech: Spontaneity Filter, Manifest and Annotations},  author    = {Yergaliyev, Askhat},  year      = {2026},  publisher = {Zenodo},  doi       = {10.5281/zenodo.23357132}}

Filter, lexicons, register-labelled manifest, evaluation scripts — 10.5281/zenodo.23357132
Source repository — github.com/AskhaYergali/kazakh-spontaneous-speech
Base model — abilmansplus/whisper-turbo-ksc2

Askhat Yergaliyev, International Information Technology University, Almaty, Kazakhstan.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

askhaYerga

Model Tree

Base

abilmansplus/whisper-turbo-ksc2

Adapter

this model

Input Modalities

Audio

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

Intended use

Not recommended for

Content-oriented downstream tasks. On a named-entity proxy, with filled pauses removed from reference and hypothesis alike, this adapter is not significantly better than the base model (F1 0.725 vs 0.705, p = 0.33). If you need entities, topics or intents rather than a verbatim transcript, the base model is enough.
Prepared or read speech. No benefit is expected; none is measured (−0.23 p.p., p = 0.25).
Languages other than Kazakh, including Kazakh–Russian code-switched speech beyond what KSC2 contains.

Usage

python

import torch, soundfile as sffrom transformers import WhisperForConditionalGeneration, WhisperProcessorfrom peft import PeftModel BASE = "abilmansplus/whisper-turbo-ksc2"ADAPTER = "askhaYerga/whisper-turbo-ksc2-kk-spontaneous-lora" processor = WhisperProcessor.from_pretrained(BASE)model = WhisperForConditionalGeneration.from_pretrained(BASE, torch_dtype=torch.float16)model = PeftModel.from_pretrained(model, ADAPTER)model = model.merge_and_unload().to("cuda").eval()   # merging is optional wav, sr = sf.read("utterance.wav")                   # 16 kHz monoinputs = processor(wav, sampling_rate=16000, return_tensors="pt")feats = inputs.input_features.to("cuda", dtype=torch.float16) # input_features passed by keyword: PeftModelForSeq2SeqLM.generate() takes no# positional arguments, so this call works with or without merge_and_unload()ids = model.generate(input_features=feats, num_beams=5, max_new_tokens=220,                     language="kazakh", task="transcribe")print(processor.batch_decode(ids, skip_special_tokens=True)[0])

import torch, soundfile as sffrom transformers import WhisperForConditionalGeneration, WhisperProcessorfrom peft import PeftModel BASE = "abilmansplus/whisper-turbo-ksc2"ADAPTER = "askhaYerga/whisper-turbo-ksc2-kk-spontaneous-lora" processor = WhisperProcessor.from_pretrained(BASE)model = WhisperForConditionalGeneration.from_pretrained(BASE, torch_dtype=torch.float16)model = PeftModel.from_pretrained(model, ADAPTER)model = model.merge_and_unload().to("cuda").eval()   # merging is optional wav, sr = sf.read("utterance.wav")                   # 16 kHz monoinputs = processor(wav, sampling_rate=16000, return_tensors="pt")feats = inputs.input_features.to("cuda", dtype=torch.float16) # input_features passed by keyword: PeftModelForSeq2SeqLM.generate() takes no# positional arguments, so this call works with or without merge_and_unload()ids = model.generate(input_features=feats, num_beams=5, max_new_tokens=220,                     language="kazakh", task="transcribe")print(processor.batch_decode(ids, skip_special_tokens=True)[0])

Training data

Training configuration

Table with columns: Hyperparameter, Value
Hyperparameter	Value
Rank r / scaling α	16 / 32
Target modules	query and value projections in self- and cross-attention, encoder and decoder
Trainable parameters	3.28 M — 0.41 % of the base model
Dropout	0.05
Learning rate / schedule	1e-4, cosine, 10 % warm-up
Effective batch size	32 (8 × 4 gradient accumulation)
Epochs	3, fixed; no checkpoint selection
Augmentation	SpecAugment, on the fly; no synthetic disfluency injection
Precision	fp16
Hardware / time	NVIDIA RTX 4090 (24 GB); ≈ 35 min per run
Adapter size	≈ 13 MB

Evaluation

Statistics. Paired bootstrap with 10,000 resamples of recording sessions, not of utterances. Decoding: beam 5, forced Kazakh language token, no external language model.

Word error rate (%)

Table with columns: Base — spont., Adapter — spont., Base — prep., Adapter — prep.
	Base — spont.	Adapter — spont.	Base — prep.	Adapter — prep.
(c) canonical	13.48	10.78 ± 0.04	12.22	11.99 ± 0.05
(a) fillers retained	17.78	15.42 ± 0.06	12.67	12.82 ± 0.05
(b) fillers removed	11.27	11.22 ± 0.04	11.67	11.47 ± 0.03

Character error rate (%)

Table with columns: Base — spont., Adapter — spont., Base — prep., Adapter — prep.
	Base — spont.	Adapter — spont.	Base — prep.	Adapter — prep.
(c) canonical	6.63	4.89 ± 0.07	4.89	4.77 ± 0.02
(a) fillers retained	7.83	6.53 ± 0.06	4.84	4.92 ± 0.02
(b) fillers removed	4.72	4.59 ± 0.07	4.53	4.43 ± 0.01

What happens to the filled pauses (%)

Table with columns: Model, Transcribed exactly, Other spelling, Omitted, Replaced by a word
Model	Transcribed exactly	Other spelling	Omitted	Replaced by a word
Base	45.6	28.7	23.0	2.7
Adapter	63.1	30.6	4.6	1.8

Reproducibility

Limitations

Filled-pause spelling is not a reliable category in Kazakh. In a blind listening audit of 30 reference/hypothesis disagreements the listener sided with the model 15 times and with the reference 15 times. Standard WER penalises a disagreement that humans share, which is why canonical scoring is used here.
Register-targeted data is not the active ingredient. A size- and duration-matched random sample of KSC2 Train recovers 93 % of the aggregate gain, and an adapter trained on prepared speech from the same recording sessions reaches the same canonical WER. What curated spontaneous data buys is a lower residual omission rate (4.6 % vs 6.2 %, p < 0.001), not a lower word error rate.
No speaker metadata. KSC2 provides none; recording sessions serve as an approximate speaker proxy, so speaker disjointness between training and evaluation is substantially reduced but not strictly guaranteed.
Third-party base model. The training normalisation of whisper-turbo-ksc2 is not documented, so the size of the fluency prior characterises that model rather than Whisper as such.
Material. Broadcast, podcast, radio, talk-show and parliamentary speech; utterances of 2.5–22 s. Behaviour on telephone-band audio, far-field recordings or long-form input is untested.
Offline transcription only. Streaming with bounded latency was not evaluated.

Licence and attribution

Released under the MIT licence, matching the base model.

Mussakhojayeva, S.; Khassanov, Y.; Varol, H.A. KSC2: An Industrial-Scale Open-Source Kazakh Speech Corpus. Proceedings of Interspeech 2022, Incheon, 2022, pp. 1367–1371. https://doi.org/10.21437/Interspeech.2022-421

Citation

bibtex

@article{yergaliyev2023spontaneous,  title   = {Spontaneous Speech and Its Features Are Taken into Account             When Creating Recognition Programs},  author  = {Yergaliyev, Askhat and Sharipbay, Altynbek and Baibulekova, Lyailya},  journal = {Ing\'enierie des Syst\`emes d'Information},  volume  = {28},  number  = {2},  pages   = {443--450},  year    = {2023},  doi     = {10.18280/isi.280220}} @dataset{yergaliyev2026manifest,  title     = {Kazakh Spontaneous Speech: Spontaneity Filter, Manifest and Annotations},  author    = {Yergaliyev, Askhat},  year      = {2026},  publisher = {Zenodo},  doi       = {10.5281/zenodo.23357132}}

@article{yergaliyev2023spontaneous,  title   = {Spontaneous Speech and Its Features Are Taken into Account             When Creating Recognition Programs},  author  = {Yergaliyev, Askhat and Sharipbay, Altynbek and Baibulekova, Lyailya},  journal = {Ing\'enierie des Syst\`emes d'Information},  volume  = {28},  number  = {2},  pages   = {443--450},  year    = {2023},  doi     = {10.18280/isi.280220}} @dataset{yergaliyev2026manifest,  title     = {Kazakh Spontaneous Speech: Spontaneity Filter, Manifest and Annotations},  author    = {Yergaliyev, Askhat},  year      = {2026},  publisher = {Zenodo},  doi       = {10.5281/zenodo.23357132}}

Filter, lexicons, register-labelled manifest, evaluation scripts — 10.5281/zenodo.23357132
Source repository — github.com/AskhaYergali/kazakh-spontaneous-speech
Base model — abilmansplus/whisper-turbo-ksc2

Askhat Yergaliyev, International Information Technology University, Almaty, Kazakhstan.

whisper-turbo-ksc2-kk-spontaneous-lora

README

Intended use

Not recommended for

Usage

Training data

Training configuration

Evaluation

Word error rate (%)

Character error rate (%)

What happens to the filled pauses (%)

Reproducibility

Limitations

Licence and attribution

Citation

Explore FriendliAI today

README

Intended use

Not recommended for

Usage

Training data

Training configuration

Evaluation

Word error rate (%)

Character error rate (%)

What happens to the filled pauses (%)

Reproducibility

Limitations

Licence and attribution

Citation

whisper-turbo-ksc2-kk-spontaneous-lora

README

Intended use

Not recommended for

Usage

Training data

Training configuration

Evaluation

Word error rate (%)

Character error rate (%)

What happens to the filled pauses (%)

Reproducibility

Limitations

Licence and attribution

Citation

Related artifacts

Explore FriendliAI today

README

Intended use

Not recommended for

Usage

Training data

Training configuration

Evaluation

Word error rate (%)

Character error rate (%)

What happens to the filled pauses (%)

Reproducibility

Limitations

Licence and attribution

Citation

Related artifacts