Paulwalker4884

whisper-persian

README

License: apache-2.0

Model Details

Model Description

This model is a fine-tuned version of OpenAI's Whisper-base model, specifically adapted for Persian language speech recognition. The model uses LoRA (Low-Rank Adaptation) for efficient fine-tuning while maintaining the original model's capabilities.

Developed by: Yasin Keykh
Model type: Speech-to-Text (Automatic Speech Recognition)
Language(s): Persian (Farsi)
License: Apache 2.0
Finetuned from model: openai/whisper-base
Fine-tuning method: LoRA (Low-Rank Adaptation)

Model Sources

Base Model: openai/whisper-base
Fine-tuning Framework: PEFT (Parameter-Efficient Fine-Tuning)

Uses

Direct Use

This model is designed for Persian speech-to-text conversion with real-time capabilities. It can be used to:

Real-time Persian speech recognition using microphone
Transcribe Persian audio files with high accuracy
Convert Persian speech to text in live applications
Build Persian voice assistants or dictation systems
Create subtitles for Persian audio/video content

Downstream Use

The model can be integrated into larger applications such as:

Voice-controlled Persian applications
Persian podcast transcription services
Educational tools for Persian language learning
Accessibility tools for Persian-speaking users

Out-of-Scope Use

The model is optimized for Persian and may not perform well on other languages
Not suitable for noisy environments without proper audio preprocessing
May have reduced accuracy on dialects significantly different from the training data

Use in Transformers

python
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq

processor = AutoProcessor.from_pretrained("Paulwalker4884/whisper-persian")
model = AutoModelForSpeechSeq2Seq.from_pretrained("Paulwalker4884/whisper-persian")

How to Get Started with the Model

Installation

First, install the required dependencies:

bash
pip install transformers torch torchaudio numpy sounddevice

Usage

Real-time Audio Recording and Transcription

python
import numpy as np
import sounddevice as sd
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
import torch

# Load the fine-tuned Persian model
processor = AutoProcessor.from_pretrained("Paulwalker4884/whisper-persian")
model = AutoModelForSpeechSeq2Seq.from_pretrained("Paulwalker4884/whisper-persian").to("cpu")

# Record audio
duration = 5  # seconds
sample_rate = 16000

print("شروع ضبط...")
audio = sd.rec(int(duration * sample_rate), samplerate=sample_rate, channels=1)
sd.wait()
print("پایان ضبط.")

# Convert to 1D array
audio = np.squeeze(audio)

# Process audio
input_features = processor(audio, sampling_rate=sample_rate, return_tensors="pt").input_features

# Generate transcription
predicted_ids = model.generate(input_features)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]

print("متن شناسایی شده:")
print(transcription)

Audio File Transcription

python
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
import torch
import torchaudio

# Load the model and processor
processor = AutoProcessor.from_pretrained("Paulwalker4884/whisper-persian")
model = AutoModelForSpeechSeq2Seq.from_pretrained("Paulwalker4884/whisper-persian")

# Load and preprocess audio
audio_path = "your_persian_audio.wav"
waveform, sample_rate = torchaudio.load(audio_path)

# Resample to 16kHz if necessary
if sample_rate != 16000:
    resampler = torchaudio.transforms.Resample(sample_rate, 16000)
    waveform = resampler(waveform)

# Process audio
input_features = processor(waveform.squeeze(), sampling_rate=16000, return_tensors="pt").input_features

# Generate transcription
with torch.no_grad():
    predicted_ids = model.generate(input_features)
    
# Decode the transcription
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(f"Transcription: {transcription}")

Batch Processing

python
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
import torch
import torchaudio

# Load the model and processor
processor = AutoProcessor.from_pretrained("Paulwalker4884/whisper-persian")
model = AutoModelForSpeechSeq2Seq.from_pretrained("Paulwalker4884/whisper-persian")

# For processing multiple audio files
def transcribe_persian_audio(audio_paths):
    transcriptions = []
    
    for audio_path in audio_paths:
        waveform, sample_rate = torchaudio.load(audio_path)
        
        if sample_rate != 16000:
            resampler = torchaudio.transforms.Resample(sample_rate, 16000)
            waveform = resampler(waveform)
        
        input_features = processor(waveform.squeeze(), sampling_rate=16000, return_tensors="pt").input_features
        
        with torch.no_grad():
            predicted_ids = model.generate(input_features)
        
        transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
        transcriptions.append(transcription)
    
    return transcriptions

# Usage
audio_files = ["audio1.wav", "audio2.wav", "audio3.wav"]
results = transcribe_persian_audio(audio_files)

Training Details

Training Data

The model was fine-tuned on Persian speech data to improve performance on Farsi language recognition tasks. The training focused on:

Common Persian vocabulary and phrases
Various Persian accents and speaking styles
Different audio qualities and recording conditions

Training Procedure

Fine-tuning Method

Technique: LoRA (Low-Rank Adaptation)
Framework: PEFT 0.17.0
Base Model: openai/whisper-base

Training Hyperparameters

Fine-tuning approach: Parameter-efficient fine-tuning with LoRA
Target modules: Attention layers and feed-forward networks
LoRA rank: 8-16 (typical range for speech models)

Evaluation

The model has been evaluated on Persian speech recognition benchmarks and shows improved performance over the base Whisper model for Persian language tasks.

Metrics

Word Error Rate (WER): Improved compared to base model on Persian test sets
Character Error Rate (CER): Enhanced character-level accuracy for Persian text

Bias, Risks, and Limitations

Limitations

Performance may vary depending on audio quality and recording conditions
Accuracy might be reduced for strong dialectal variations
May have lower performance on technical or domain-specific Persian terminology not present in training data

Recommendations

Ensure good audio quality for optimal performance
Consider audio preprocessing for noisy environments
Test the model on your specific use case to evaluate performance
Be aware of potential biases in training data that may affect certain speakers or contexts

Technical Specifications

Model Architecture

Base Architecture: Whisper Transformer
Fine-tuning Method: LoRA adapters
Input: 16kHz mono audio
Output: Persian text transcription

Framework Versions

PEFT: 0.17.0
Transformers: Compatible with latest versions
PyTorch: 1.9.0+

Citation

If you use this model in your research or applications, please cite:

bibtex
@misc{whisper-persian-paulwalker4884,
  author = {Yasin Keykh},
  title = {Whisper Persian Fine-tuned Model},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Paulwalker4884/whisper-persian}
}

Model Card Contact

Author: Yasin Keykh

For questions or issues regarding this model, please open an issue in the model repository or contact the author directly.

This model is based on OpenAI's Whisper and has been fine-tuned for Persian language speech recognition using modern parameter-efficient fine-tuning techniques.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Paulwalker4884

Model Tree

Base

openai/whisper-base

Adapter

this model

Input Modalities

Audio

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Details

Model Description

Developed by: Yasin Keykh
Model type: Speech-to-Text (Automatic Speech Recognition)
Language(s): Persian (Farsi)
License: Apache 2.0
Finetuned from model: openai/whisper-base
Fine-tuning method: LoRA (Low-Rank Adaptation)

Model Sources

Base Model: openai/whisper-base
Fine-tuning Framework: PEFT (Parameter-Efficient Fine-Tuning)

Uses

Direct Use

This model is designed for Persian speech-to-text conversion with real-time capabilities. It can be used to:

Real-time Persian speech recognition using microphone
Transcribe Persian audio files with high accuracy
Convert Persian speech to text in live applications
Build Persian voice assistants or dictation systems
Create subtitles for Persian audio/video content

Downstream Use

The model can be integrated into larger applications such as:

Voice-controlled Persian applications
Persian podcast transcription services
Educational tools for Persian language learning
Accessibility tools for Persian-speaking users

Out-of-Scope Use

The model is optimized for Persian and may not perform well on other languages
Not suitable for noisy environments without proper audio preprocessing
May have reduced accuracy on dialects significantly different from the training data

Use in Transformers

python
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq

processor = AutoProcessor.from_pretrained("Paulwalker4884/whisper-persian")
model = AutoModelForSpeechSeq2Seq.from_pretrained("Paulwalker4884/whisper-persian")

How to Get Started with the Model

Installation

First, install the required dependencies:

bash
pip install transformers torch torchaudio numpy sounddevice

Usage

Real-time Audio Recording and Transcription

python
import numpy as np
import sounddevice as sd
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
import torch

# Load the fine-tuned Persian model
processor = AutoProcessor.from_pretrained("Paulwalker4884/whisper-persian")
model = AutoModelForSpeechSeq2Seq.from_pretrained("Paulwalker4884/whisper-persian").to("cpu")

# Record audio
duration = 5  # seconds
sample_rate = 16000

print("شروع ضبط...")
audio = sd.rec(int(duration * sample_rate), samplerate=sample_rate, channels=1)
sd.wait()
print("پایان ضبط.")

# Convert to 1D array
audio = np.squeeze(audio)

# Process audio
input_features = processor(audio, sampling_rate=sample_rate, return_tensors="pt").input_features

# Generate transcription
predicted_ids = model.generate(input_features)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]

print("متن شناسایی شده:")
print(transcription)

Audio File Transcription

python
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
import torch
import torchaudio

# Load the model and processor
processor = AutoProcessor.from_pretrained("Paulwalker4884/whisper-persian")
model = AutoModelForSpeechSeq2Seq.from_pretrained("Paulwalker4884/whisper-persian")

# Load and preprocess audio
audio_path = "your_persian_audio.wav"
waveform, sample_rate = torchaudio.load(audio_path)

# Resample to 16kHz if necessary
if sample_rate != 16000:
    resampler = torchaudio.transforms.Resample(sample_rate, 16000)
    waveform = resampler(waveform)

# Process audio
input_features = processor(waveform.squeeze(), sampling_rate=16000, return_tensors="pt").input_features

# Generate transcription
with torch.no_grad():
    predicted_ids = model.generate(input_features)
    
# Decode the transcription
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(f"Transcription: {transcription}")

Batch Processing

python
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
import torch
import torchaudio

# Load the model and processor
processor = AutoProcessor.from_pretrained("Paulwalker4884/whisper-persian")
model = AutoModelForSpeechSeq2Seq.from_pretrained("Paulwalker4884/whisper-persian")

# For processing multiple audio files
def transcribe_persian_audio(audio_paths):
    transcriptions = []
    
    for audio_path in audio_paths:
        waveform, sample_rate = torchaudio.load(audio_path)
        
        if sample_rate != 16000:
            resampler = torchaudio.transforms.Resample(sample_rate, 16000)
            waveform = resampler(waveform)
        
        input_features = processor(waveform.squeeze(), sampling_rate=16000, return_tensors="pt").input_features
        
        with torch.no_grad():
            predicted_ids = model.generate(input_features)
        
        transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
        transcriptions.append(transcription)
    
    return transcriptions

# Usage
audio_files = ["audio1.wav", "audio2.wav", "audio3.wav"]
results = transcribe_persian_audio(audio_files)

Training Details

Training Data

The model was fine-tuned on Persian speech data to improve performance on Farsi language recognition tasks. The training focused on:

Common Persian vocabulary and phrases
Various Persian accents and speaking styles
Different audio qualities and recording conditions

Training Procedure

Fine-tuning Method

Technique: LoRA (Low-Rank Adaptation)
Framework: PEFT 0.17.0
Base Model: openai/whisper-base

Training Hyperparameters

Fine-tuning approach: Parameter-efficient fine-tuning with LoRA
Target modules: Attention layers and feed-forward networks
LoRA rank: 8-16 (typical range for speech models)

Evaluation

The model has been evaluated on Persian speech recognition benchmarks and shows improved performance over the base Whisper model for Persian language tasks.

Metrics

Word Error Rate (WER): Improved compared to base model on Persian test sets
Character Error Rate (CER): Enhanced character-level accuracy for Persian text

Bias, Risks, and Limitations

Limitations

Performance may vary depending on audio quality and recording conditions
Accuracy might be reduced for strong dialectal variations
May have lower performance on technical or domain-specific Persian terminology not present in training data

Recommendations

Ensure good audio quality for optimal performance
Consider audio preprocessing for noisy environments
Test the model on your specific use case to evaluate performance
Be aware of potential biases in training data that may affect certain speakers or contexts

Technical Specifications

Model Architecture

Base Architecture: Whisper Transformer
Fine-tuning Method: LoRA adapters
Input: 16kHz mono audio
Output: Persian text transcription

Framework Versions

PEFT: 0.17.0
Transformers: Compatible with latest versions
PyTorch: 1.9.0+

Citation

If you use this model in your research or applications, please cite:

bibtex
@misc{whisper-persian-paulwalker4884,
  author = {Yasin Keykh},
  title = {Whisper Persian Fine-tuned Model},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Paulwalker4884/whisper-persian}
}

Model Card Contact

Author: Yasin Keykh

For questions or issues regarding this model, please open an issue in the model repository or contact the author directly.

This model is based on OpenAI's Whisper and has been fine-tuned for Persian language speech recognition using modern parameter-efficient fine-tuning techniques.