GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Akash-Sakala
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Akash-Sakala
Model Tree
openai/whisper-large-v3
Input Modalities
Output Modalities
Supported Functionality
| Base model | openai/whisper-large-v3 (1.55B params, 128 mel bins) |
| Method | QLoRA: base model frozen in 4-bit NF4 with double quantisation (fp16 compute), LoRA adapters trained in fp16 |
| LoRA rank / alpha | r = 32, α = 64 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, v_proj |
| Trainable params | ~15.7M (~1% of base) |
| Language / task | English, transcribe |
| Developed by | Akash Sakala |
| Hyperparameter | Value |
|---|---|
| Data | Svarah, ~90/10 train/eval split (~6,000 train clips) |
| Steps | 500 (≈1.33 epochs) |
| Effective batch | 16 (8 × grad-accum 2) |
| Learning rate | 1e-3, linear decay, 50 warm-up steps |
| Optimizer | AdamW (fused) |
| Precision | fp16 mixed + gradient checkpointing |
| Best checkpoint | lowest eval loss (load_best_model_at_end) |
| Wall-clock | ~94 min, single GPU |
| Step | Train loss | Eval loss |
|---|---|---|
| 100 | 0.246 | 0.255 |
| 200 | 0.238 | 0.244 |
| 300 | 0.225 | 0.219 |
| 400 | 0.140 | 0.205 |
| 500 | 0.109 | 0.202 |
Eval loss fell 21% between steps 100 and 500 (0.255 → 0.202). After the first epoch (step 375), train loss drops sharply while eval loss flattens, so 500 steps is about where overfitting starts. The TensorBoard logs are in runs/.
Word error rate (WER) on the held-out split hasn't been measured yet. Eval loss is the only held-out metric reported here.
python
import torch, librosafrom transformers import WhisperForConditionalGeneration, WhisperProcessorfrom peft import PeftModel processor = WhisperProcessor.from_pretrained("openai/whisper-large-v3")base = WhisperForConditionalGeneration.from_pretrained( "openai/whisper-large-v3", torch_dtype=torch.float16, device_map="auto")model = PeftModel.from_pretrained(base, "Akash-Sakala/whisper-large-v3-svarah-indic")model = model.merge_and_unload().eval() audio, _ = librosa.load("sample.wav", sr=16_000)features = processor(audio, sampling_rate=16_000, return_tensors="pt").input_featuresfeatures = features.to(model.device, torch.float16) with torch.no_grad(): ids = model.generate(features, language="en", task="transcribe")print(processor.batch_decode(ids, skip_special_tokens=True)[0])Low-VRAM option: load the base model with BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4") as it was during training, and skip merge_and_unload().
Akash-Sakala/whisper-small-emotion: a LoRA for Whisper-small on Indian-English expressive speech (WER 9.16% → 4.51%)Akash-Sakala/qwen3.5-4b-transcript-formatter-lora: an ASR post-processor that formats raw transcriptsbibtex
@inproceedings{javed2023svarah, title = {Svarah: Evaluating English ASR Systems on Indian Accents}, author = {Javed, Tahir and Joshi, Sakshi and Nagarajan, Vignesh and Sundaresan, Sai and Nawale, Janki and Raman, Abhigyan and Bhogale, Kaushal and Kumar, Pratyush and Khapra, Mitesh M.}, journal = {arXiv preprint arXiv:2305.15760}, year = {2023}}import torch, librosafrom transformers import WhisperForConditionalGeneration, WhisperProcessorfrom peft import PeftModel processor = WhisperProcessor.from_pretrained("openai/whisper-large-v3")base = WhisperForConditionalGeneration.from_pretrained( "openai/whisper-large-v3", torch_dtype=torch.float16, device_map="auto")model = PeftModel.from_pretrained(base, "Akash-Sakala/whisper-large-v3-svarah-indic")model = model.merge_and_unload().eval() audio, _ = librosa.load("sample.wav", sr=16_000)features = processor(audio, sampling_rate=16_000, return_tensors="pt").input_featuresfeatures = features.to(model.device, torch.float16) with torch.no_grad(): ids = model.generate(features, language="en", task="transcribe")print(processor.batch_decode(ids, skip_special_tokens=True)[0])@inproceedings{javed2023svarah, title = {Svarah: Evaluating English ASR Systems on Indian Accents}, author = {Javed, Tahir and Joshi, Sakshi and Nagarajan, Vignesh and Sundaresan, Sai and Nawale, Janki and Raman, Abhigyan and Bhogale, Kaushal and Kumar, Pratyush and Khapra, Mitesh M.}, journal = {arXiv preprint arXiv:2305.15760}, year = {2023}}