MediBeng-Whisper-Tiny-FL

Model Description

MediBeng Whisper Tiny FL is the federated learning (v3) release of Whisper Tiny for automatic speech translation of code-switched Bengali–English clinical conversations into English. Training uses FedProx across 4 simulated hospital clients with speaker-based non-IID data partitions — raw audio never leaves each client; only aggregated weight updates are shared.

This release is the best-performing FL checkpoint (FedProx speaker). It substantially outperforms centralised v2 under the same free-generation evaluation protocol.

What’s New in v3 (Federated Learning)

Federated architecture

markdown
Copy code
┌─────────────────────────┐
                 │   FL Server (Aggregator) │
                 │   FedProx (μ = 0.01)     │
                 └────────────┬────────────┘
                              │ broadcast global weights
        ┌─────────────────────┼─────────────────────┐
        │                     │                     │
 ┌──────▼──────┐      ┌──────▼──────┐      ┌──────▼──────┐
 │  Client 0   │      │  Client 1   │ ...  │  Client 3   │
 │ Male shard  │      │ Male shard  │      │ Female shard│
 │ 863 samples │      │ 863 samples │      │ 843 samples │
 │ 50 local    │      │ 50 local    │      │ 50 local    │
 │ steps/round │      │ steps/round │      │ steps/round │
 └──────┬──────┘      └──────┬──────┘      └──────┬──────┘
        └─────────────────────┼─────────────────────┘
                              │ upload weight updates
                 ┌────────────▼────────────┐
                 │  Weighted Aggregation   │
                 └─────────────────────────┘

Training configuration

Usage

Install dependencies:

bash
Copy code
pip install transformers librosa torch datasets

Run inference (free generation — matches evaluation protocol):

python
Copy code
from transformers import WhisperProcessor, WhisperForConditionalGeneration
import librosa

MODEL_ID = "pr0mila-gh0sh/MediBeng-Whisper-Tiny-FL"

processor = WhisperProcessor.from_pretrained(MODEL_ID)
model = WhisperForConditionalGeneration.from_pretrained(MODEL_ID)

# Free generation (canonical FL evaluation protocol)
model.config.forced_decoder_ids = None
model.generation_config.forced_decoder_ids = None

audio, _ = librosa.load("path_to_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features

predicted_ids = model.generate(input_features, max_length=225)
translation = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]

print("Translation:", translation)

Intended Use

For researchers and developers building privacy-preserving clinical AST systems:

• Multi-hospital deployment without centralising patient audio
• Federated fine-tuning research on code-switched clinical speech
• Bengali–English medical translation in regulated environments

Training Data

Fine-tuned via federated learning on the MediBeng training split, partitioned across 4 simulated clients:

Test evaluation: held-out 960-sample HF test split (identical to v2).

Evaluation Results

Full test set (n = 960, free generation)

FedProx convergence (quick eval, n = 200 per round)

Statistical significance (FedProx vs baseline)

Limitations

• Simulated clients — partitions use synthetic TTS speaker labels, not real multi-hospital data.
• 3 FL rounds — convergence may improve with more rounds.
• Centralised upper bound — comparison vs under-trained centralised v2; fully converged centralised baseline pending.
• Full weight transmission — ~156 MB per client per round; LoRA-FL not yet implemented.
• No differential privacy — formal ε–δ guarantees not yet added.

Ethical Considerations

• Federated learning reduces raw-data exposure but does not eliminate all privacy risks (model inversion, membership inference).
• Training data may reflect demographic biases; validate before clinical deployment.
• Human review required for all clinical translations.

Blog Post

MediBeng Whisper-Tiny: Translating Code-Switched Bengali-English Speech for Healthcare

Citation

Preprint on medRxiv.

bibtex
Copy code
@article{ghosh2025medibeng,
  title={MediBeng Whisper Tiny: A fine-tuned code-switched Bengali-English translator for clinical applications},
  author={Ghosh, Promila and Talukder, Sunipun},
  journal={medRxiv},
  year={2025},
  doi={https://doi.org/10.1101/2025.04.25.25326406},
  url={https://www.medrxiv.org/content/10.1101/2025.04.25.25326406v2}
}