Model description
This model is a fine-tuned version of openai/whisper-large-v3, adapted using Parameter-Efficient Fine-Tuning (PEFT) with Low-Rank Adaptation (LoRA) for Automatic Speech Recognition (ASR) of the low-resource Tboli language.
It is designed to transcribe spoken Tboli words and short phrases into text using a predefined vocabulary. The model serves as the speech recognition component of a Tboli-to-English speech translation system.
This repository contains LoRA adapter weights intended to be used with the original Whisper Large V3 base model.
Intended uses & limitations
The model is intended for:
- Transcribing Tboli speech within its trained vocabulary.
- Research and experimentation in low-resource speech recognition.
- Integration with a machine translation model for Tboli-to-English translation.
Limitations:
- Recognition is primarily limited to the vocabulary represented in the training dataset.
- Performance may vary with unseen words, continuous speech, different accents, and noisy recordings.
- The model performs speech-to-text transcription only and does not directly translate Tboli into English.
Training and evaluation data
The model was fine-tuned on a privately maintained Tboli speech dataset containing audio recordings of predefined words and short phrases paired with their corresponding transcriptions.
Audio recordings were preprocessed and resampled to 16 kHz. Data augmentation techniques, including noise, pitch, and speed variations, were applied to the training samples.
The dataset was divided into training, validation, and testing subsets using an 80/10/10 split. Evaluation was conducted using Character Error Rate (CER) and Word Error Rate (WER).
Dataset availability: The datset is private and not publicly available. The release of this model does not grant access to, ownership of, or permission to redistribute the underlying Tboli speech dataset.
Last logged training evaluation
- step: 2500
- training loss: 1.9303
- validation loss: 0.4578
- cer: 15.93%
- wer: 57.50%
- cer: 15.06%
- wer: 50.42%
- mean cer ± sd: 14.73 ± 20.21
- mean wer ± sd: 46.47 ± 50.56
These results reflect speech recognition performance within the predefined Tboli vocabulary used during training and evaluation.
Training procedure
Fine-tuning method
Fine-tuning method
- The model was adapted using PEFT with LoRA.
- LoRA rank: 16
- LoRA alpha: 32
- LoRA dropout: 0.05
- Target modules: q_proj, v_proj
- Tokenizer language: Tagalog (tl) as a proxy for Tboli
- Task: Transcription
Training hyperparameters
The following hyperparameters were used during training:
- Learning rate: 5e-6
- Training batch size: 8
- Evaluation batch size: 4
- Gradient accumulation steps: 4
- Effective training batch size: 32
- Random seed: 42
- Optimizer: AdamW (Torch Fused)
- Learning rate scheduler: Linear
- Warmup steps: 100
- Training epochs: 12
- Mixed precision: FP16
- Gradient checkpointing: Enabled
Framework versions
- Transformers 5.0.0
- Pytorch 2.9.0+cu126
- Datasets 4.5.0
- Tokenizers 0.22.2
Evaluation considerations
Given the limited vocabulary coverage and the recording-level dataset split, additional evaluations would be necessary to establish model generalization beyond the predefined research corpus.
Future evaluation should consider unseen vocabulary, continuous speech, additional native speakers, and more diverse acoustic conditions.
The dataset and its evaluation results should be understood as an initial contribution toward low-resource Tboli speech recognition research rather than a comprehensive representation of the Tboli language.