GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
milanakdj
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
milanakdj
Model Tree
openai/whisper-small
Input Modalities
Output Modalities
Supported Functionality
This model is designed to transcribe Tibetan speech into written text. It can be used for:
The model was trained on the Merged Tibetan Titung Goose dataset, which combines multiple Tibetan audio sources for improved coverage.
lilgoose777/merged-tibetan-titung-gooseThe model was fine-tuned with the following configuration:
| Hyperparameter | Value |
|---|---|
| Base Model | openai/whisper-small |
| Training Steps | 3,000 |
| Batch Size (Train) | 16 |
| Batch Size (Eval) | 8 |
| Learning Rate | 1.25e-05 |
| Warmup Steps | 300 |
| Gradient Accumulation | 1 |
| Gradient Checkpointing | True |
| Mixed Precision (FP16) | False |
| Max Generation Length | 225 |
| Evaluation Strategy | Every 100 steps |
| Save Strategy | Every 100 steps |
| Metric | Value |
|---|---|
| Best WER | XX.XX% |
| Final WER | XX.XX% |
Lower WER is better. WER (Word Error Rate) measures the percentage of words that are incorrectly transcribed.
python
from transformers import pipeline # Create transcription pipelinepipe = pipeline( "automatic-speech-recognition", model="milanakdj/whisper-small-full-tibetan", generate_kwargs={"language": "tibetan", "task": "translate"}) # Transcribe audio fileresult = pipe("path/to/tibetan_audio.wav")print(result["text"])python
from transformers import WhisperProcessor, WhisperForConditionalGenerationimport torch # Load model and processorprocessor = WhisperProcessor.from_pretrained("milanakdj/whisper-small-full-tibetan")model = WhisperForConditionalGeneration.from_pretrained("milanakdj/whisper-small-full-tibetan") # Load your audio (16kHz sampling rate)# audio_array = ... your audio array ... # Process audioinput_features = processor( audio_array, sampling_rate=16000, return_tensors="pt").input_features # Generate transcriptionforced_decoder_ids = processor.get_decoder_prompt_ids( language="tibetan", task="translate") with torch.no_grad(): predicted_ids = model.generate( input_features, forced_decoder_ids=forced_decoder_ids ) # Decodetranscription = processor.batch_decode( predicted_ids, skip_special_tokens=True)[0] print(transcription)python
import librosafrom transformers import WhisperProcessor, WhisperForConditionalGeneration # Load modelprocessor = WhisperProcessor.from_pretrained("milanakdj/whisper-small-full-tibetan")model = WhisperForConditionalGeneration.from_pretrained("milanakdj/whisper-small-full-tibetan") # Load audio file (automatically resamples to 16kHz)audio, sr = librosa.load("tibetan_audio.mp3", sr=16000) # Process and transcribeinput_features = processor( audio, sampling_rate=16000, return_tensors="pt").input_features forced_decoder_ids = processor.get_decoder_prompt_ids( language="tibetan", task="translate") predicted_ids = model.generate( input_features, forced_decoder_ids=forced_decoder_ids) transcription = processor.batch_decode( predicted_ids, skip_special_tokens=True)[0] print(transcription)python
import torchimport librosafrom transformers import WhisperProcessor, WhisperForConditionalGeneration # Load modelprocessor = WhisperProcessor.from_pretrained("milanakdj/whisper-small-full-tibetan")model = WhisperForConditionalGeneration.from_pretrained("milanakdj/whisper-small-full-tibetan") # Set devicedevice = "cuda" if torch.cuda.is_available() else "cpu"model.to(device) # Process multiple filesaudio_files = ["file1.wav", "file2.wav", "file3.wav"] for audio_file in audio_files: # Load audio audio, _ = librosa.load(audio_file, sr=16000) # Process input_features = processor( audio, sampling_rate=16000, return_tensors="pt" ).input_features.to(device) # Generate forced_decoder_ids = processor.get_decoder_prompt_ids( language="tibetan", task="translate" ) with torch.no_grad(): predicted_ids = model.generate( input_features, forced_decoder_ids=forced_decoder_ids ) # Decode transcription = processor.batch_decode( predicted_ids, skip_special_tokens=True )[0] print(f"{audio_file}: {transcription}")To evaluate the model on your own Tibetan audio dataset:
python
from transformers import WhisperProcessor, WhisperForConditionalGenerationfrom datasets import load_datasetimport evaluate # Load modelprocessor = WhisperProcessor.from_pretrained("milanakdj/whisper-small-full-tibetan")model = WhisperForConditionalGeneration.from_pretrained("milanakdj/whisper-small-full-tibetan") # Load your datasetdataset = load_dataset("your_tibetan_dataset") # Initialize WER metricwer_metric = evaluate.load("wer") # Process and evaluate# ... (see full evaluation code in documentation)This model is intended for:
If you use this model in your research or application, please cite:
bibtex
@misc{whisper-small-tibetan-2024, author = {Milan Akdj}, title = {Whisper Small - Tibetan Speech Translation}, year = {2024}, publisher = {HuggingFace}, journal = {HuggingFace Model Hub}, howpublished = {\url{https://huggingface.co/milanakdj/whisper-small-full-tibetan}}}Also cite the original Whisper paper:
bibtex
@article{radford2022whisper, title={Robust Speech Recognition via Large-Scale Weak Supervision}, author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya}, journal={arXiv preprint arXiv:2212.04356}, year={2022}}Milan Akdj
For questions or issues with this model, please open an issue on the model repository.
Whisper Small uses a Transformer encoder-decoder architecture:
Language: Tibetan (བོད་སྐད།)
License: Apache 2.0
Model Size: ~244M parameters
from transformers import pipeline # Create transcription pipelinepipe = pipeline( "automatic-speech-recognition", model="milanakdj/whisper-small-full-tibetan", generate_kwargs={"language": "tibetan", "task": "translate"}) # Transcribe audio fileresult = pipe("path/to/tibetan_audio.wav")print(result["text"])from transformers import WhisperProcessor, WhisperForConditionalGenerationimport torch # Load model and processorprocessor = WhisperProcessor.from_pretrained("milanakdj/whisper-small-full-tibetan")model = WhisperForConditionalGeneration.from_pretrained("milanakdj/whisper-small-full-tibetan") # Load your audio (16kHz sampling rate)# audio_array = ... your audio array ... # Process audioinput_features = processor( audio_array, sampling_rate=16000, return_tensors="pt").input_features # Generate transcriptionforced_decoder_ids = processor.get_decoder_prompt_ids( language="tibetan", task="translate") with torch.no_grad(): predicted_ids = model.generate( input_features, forced_decoder_ids=forced_decoder_ids ) # Decodetranscription = processor.batch_decode( predicted_ids, skip_special_tokens=True)[0] print(transcription)import librosafrom transformers import WhisperProcessor, WhisperForConditionalGeneration # Load modelprocessor = WhisperProcessor.from_pretrained("milanakdj/whisper-small-full-tibetan")model = WhisperForConditionalGeneration.from_pretrained("milanakdj/whisper-small-full-tibetan") # Load audio file (automatically resamples to 16kHz)audio, sr = librosa.load("tibetan_audio.mp3", sr=16000) # Process and transcribeinput_features = processor( audio, sampling_rate=16000, return_tensors="pt").input_features forced_decoder_ids = processor.get_decoder_prompt_ids( language="tibetan", task="translate") predicted_ids = model.generate( input_features, forced_decoder_ids=forced_decoder_ids) transcription = processor.batch_decode( predicted_ids, skip_special_tokens=True)[0] print(transcription)import torchimport librosafrom transformers import WhisperProcessor, WhisperForConditionalGeneration # Load modelprocessor = WhisperProcessor.from_pretrained("milanakdj/whisper-small-full-tibetan")model = WhisperForConditionalGeneration.from_pretrained("milanakdj/whisper-small-full-tibetan") # Set devicedevice = "cuda" if torch.cuda.is_available() else "cpu"model.to(device) # Process multiple filesaudio_files = ["file1.wav", "file2.wav", "file3.wav"] for audio_file in audio_files: # Load audio audio, _ = librosa.load(audio_file, sr=16000) # Process input_features = processor( audio, sampling_rate=16000, return_tensors="pt" ).input_features.to(device) # Generate forced_decoder_ids = processor.get_decoder_prompt_ids( language="tibetan", task="translate" ) with torch.no_grad(): predicted_ids = model.generate( input_features, forced_decoder_ids=forced_decoder_ids ) # Decode transcription = processor.batch_decode( predicted_ids, skip_special_tokens=True )[0] print(f"{audio_file}: {transcription}")from transformers import WhisperProcessor, WhisperForConditionalGenerationfrom datasets import load_datasetimport evaluate # Load modelprocessor = WhisperProcessor.from_pretrained("milanakdj/whisper-small-full-tibetan")model = WhisperForConditionalGeneration.from_pretrained("milanakdj/whisper-small-full-tibetan") # Load your datasetdataset = load_dataset("your_tibetan_dataset") # Initialize WER metricwer_metric = evaluate.load("wer") # Process and evaluate# ... (see full evaluation code in documentation)@misc{whisper-small-tibetan-2024, author = {Milan Akdj}, title = {Whisper Small - Tibetan Speech Translation}, year = {2024}, publisher = {HuggingFace}, journal = {HuggingFace Model Hub}, howpublished = {\url{https://huggingface.co/milanakdj/whisper-small-full-tibetan}}}@article{radford2022whisper, title={Robust Speech Recognition via Large-Scale Weak Supervision}, author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya}, journal={arXiv preprint arXiv:2212.04356}, year={2022}}