GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Daksh159
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Daksh159
Model Tree
openai/whisper-small
Input Modalities
Output Modalities
Supported Functionality
VaaniAI is a Hindi Automatic Speech Recognition (ASR) model fine-tuned from OpenAI's Whisper-small architecture.
The model converts spoken Hindi audio into written Hindi text. It is designed to explore improved Hindi transcription through domain-specific fine-tuning of a pretrained multilingual speech-recognition model.
VaaniAI is built using the Hugging Face Transformers library and can be loaded directly using its model repository.
openai/whisper-smallVaaniAI is intended for:
VaaniAI uses the Whisper-small encoder-decoder Transformer architecture.
The encoder processes audio features and produces contextual representations. The decoder generates the corresponding text transcription.
The model uses:
VaaniAI was fine-tuned from openai/whisper-small using Hindi speech recordings.
The training dataset consisted of Hindi recordings from the Josh Talks dataset.
| Dataset detail | Value |
|---|---|
| Original recordings | 104 |
| Original audio duration | 21.89 hours |
| Usable audio segments | 4,442 |
| Usable audio duration | 11.44 hours |
| Number of speakers | 102 |
| Training segments | 4,093 |
| Validation segments | 349 |
The audio was processed into segments suitable for Whisper fine-tuning.
Dataset access and licensing: The original recordings are not included in this repository. Dataset redistribution and model-weight release permissions must be confirmed with the relevant data owner.
| Parameter | Value |
|---|---|
| Base model | Whisper-small |
| Epochs | 3 |
| Training steps | 192 |
| Batch size | 4 |
| Gradient accumulation steps | 8 |
| Learning rate | 1e-5 |
| Precision | FP16 |
| Gradient checkpointing | Enabled |
| Training platform | Kaggle |
| GPU | CUDA-enabled GPU |
The original training notebook recorded the following results:
| Metric | Recorded result |
|---|---|
| Validation WER — Epoch 1 | 0.5464 |
| Validation WER — Epoch 2 | 0.4357 |
| Validation WER — Epoch 3 | 0.4013 |
| Original Whisper baseline WER | 1.2537 |
The baseline result was recorded on a 100-sample subset. These historical measurements should not be treated as a definitive comparative benchmark until the evaluation samples, normalization and methodology are reproduced consistently.
Additional evaluation on independently verified Hindi recordings is in progress.
No claim of a specific percentage improvement is made from the current small test set.
bash
pip install torch transformers librosa soundfileThe model can be loaded directly from Hugging Face.
python
import torchimport librosa from transformers import ( WhisperProcessor, WhisperForConditionalGeneration) MODEL_ID = "Daksh159/VaaniAI" processor = WhisperProcessor.from_pretrained(MODEL_ID) model = WhisperForConditionalGeneration.from_pretrained(MODEL_ID) device = "cuda" if torch.cuda.is_available() else "cpu" model = model.to(device)model.eval() audio, sr = librosa.load( "hindi_audio.mp3", sr=16000, mono=True) inputs = processor( audio, sampling_rate=16000, return_tensors="pt").input_features.to(device) with torch.no_grad(): predicted_ids = model.generate( inputs, language="hindi", task="transcribe", max_new_tokens=225 ) transcription = processor.batch_decode( predicted_ids, skip_special_tokens=True)[0] print("VaaniAI Transcription:")print(transcription)Users should manually verify transcriptions before using them in legal, medical, financial or other high-stakes applications.
Users should obtain appropriate consent before recording or processing someone's speech and should handle audio recordings and transcripts responsibly.
https://huggingface.co/Daksh159/VaaniAI
Maintainer: Daksh Goyal
import torchimport librosa from transformers import ( WhisperProcessor, WhisperForConditionalGeneration) MODEL_ID = "Daksh159/VaaniAI" processor = WhisperProcessor.from_pretrained(MODEL_ID) model = WhisperForConditionalGeneration.from_pretrained(MODEL_ID) device = "cuda" if torch.cuda.is_available() else "cpu" model = model.to(device)model.eval() audio, sr = librosa.load( "hindi_audio.mp3", sr=16000, mono=True) inputs = processor( audio, sampling_rate=16000, return_tensors="pt").input_features.to(device) with torch.no_grad(): predicted_ids = model.generate( inputs, language="hindi", task="transcribe", max_new_tokens=225 ) transcription = processor.batch_decode( predicted_ids, skip_special_tokens=True)[0] print("VaaniAI Transcription:")print(transcription)