Model Description
The model took about nine hours to train on a single A100 GPU.
⚠️ Production Status: This specific artifact is an experimental research iteration designed to analyze training dynamics and is not production-ready. The fully optimized, production-grade model with mitigated overfitting remains private.
Intended Uses & Limitations
This repository serves as a benchmark checkpoint for studying fine-tuning constraints on Turkish ASR. The primary objective was increasing Turkish labeled data exposure. However, because the dataset contains only 49.945 hours of data (roughly 1.1% of the original Whisper pre-training distribution), the model exhibits standard fine-tuning constraints under out-of-domain distribution shifts.
Post-Evaluation Insights (Generalization Metrics)
Independent diagnostic evaluations reveal a high variance between clean in-domain data and noisy out-of-domain speech:
- In-Domain Memorization (FLEURS Train Split): Natively matches audio profiles with an exceptional 1.15% WER.
- Out-of-Domain Performance (MediaSpeech/OpenSLR): Degrades to a 30.29% WER due to acoustic domain mismatches (broadcast TV/radio vs. clean reading speech).
Strategic Takeaway: The training logs show clear optimization divergence past Step 2000. Beyond this threshold, the model ceased mapping generalized Turkish phonetics and began memorizing the acoustic signatures of the training subset. For downstream applications, weights should be pulled from the Step 2000 checkpoint to maintain general zero-shot capacity.
Training Procedure
Training Hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- training_steps: 4000
- mixed_precision_training: Native AMP
Training Results
Table with columns: Training Loss, Epoch, Step, Validation Loss, Wer| Training Loss | Epoch | Step | Validation Loss | Wer |
|---|
| 0.1803 | 0.36 | 1000 | 0.2089 | 18.6326 |
| 0.1428 | 0.71 | 2000 | 0.1821 | 16.3912 |
| 0.0535 | 1.07 | 3000 | 0.1693 | 14.9132 |
| 0.0491 |
Framework Versions
- Transformers 4.38.1
- Pytorch 2.1.0+cu121
- Datasets 2.17.1
- Tokenizers 0.15.2
