GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
RyanJJaison
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
RyanJJaison
Model Tree
openai/whisper-small
Input Modalities
Output Modalities
Supported Functionality
openai/whisper-small (244M params)r=32, alpha=64, dropout=0.05, bias=none,
target_modules=[q_proj, v_proj] — 3,538,944 trainable parameters
(1.44% of the base model)trp language token, so training and
inference use bn (Bengali) as the decoder language. This is a
script/tokenizer match, not a claim of phonetic similarity — Kokborok
transcripts in this corpus are written in Bengali script, so the
Bengali proxy gives the tokenizer a sane per-word subword count instead of
falling back to raw bytes.Seq2SeqTrainerBecause every split shares the same speaker, reported WER/CER are speaker-optimistic: the model can partly learn this narrator's voice rather than only Kokborok in general. Expect materially worse performance on any other speaker. This is the single biggest caveat on these numbers — see Limitations.
Metrics from results/metrics.jsonl, scored with jiwer (corpus-level WER/CER,
computed as total edits over total reference length, not averaged per utterance).
| Model | Split | WER | CER |
|---|---|---|---|
| Whisper-small, zero-shot (no adapter) | test | 1.3298 | 1.3697 |
| This adapter (final checkpoint, step 220 / epoch 20) | val | 0.9658 | 0.5749 |
| This adapter (final checkpoint, step 220 / epoch 20) | test | 1.0922 | 0.7183 |
For reference, the untuned base model achieves 0 correct words out of 282 on the test set (every hypothesis is a repetition-loop artifact or a wrong-language guess); this adapter recovers real lexical content, cuts CER by roughly half, and eliminates the repetition loops — but WER above 1.0 means the model still inserts and substitutes more than it gets exactly right.
This corpus overfits fast: validation loss bottoms out around epoch 7 and rises afterward, while validation WER (measured by a full generate pass, not loss) keeps improving to roughly epoch 14–15 before flattening. The checkpoint published here is the final epoch (20), not the validation-WER-selected checkpoint — if you are comparing against reports elsewhere for this project that cite WER 1.1348 (test) / 0.9288 (val), those numbers are for an earlier checkpoint (step 160) that scored best on validation WER at the time. Both checkpoints are close, unstably so, given the test set is only 35 utterances (a bootstrap confidence interval on numbers this size would be wide).
bn language proxy. Decoder behavior (punctuation, casing conventions)
inherits some biases from Bengali rather than Kokborok specifically.python
from transformers import WhisperForConditionalGeneration, WhisperProcessorfrom peft import PeftModelimport librosabase = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small")model = PeftModel.from_pretrained(base, "RyanJJaison/kokborok-whisper-lora")processor = WhisperProcessor.from_pretrained("RyanJJaison/kokborok-whisper-lora", language="bn", task="transcribe")audio, sr = librosa.load("clip.wav", sr=16000, mono=True)inputs = processor(audio, sampling_rate=sr, return_tensors="pt")predicted_ids = model.generate(**inputs)print(processor.batch_decode(predicted_ids, skip_special_tokens=True)[0])
asr/)asr/requirements.txt for exact pin)No formal citation; if this is useful, please link back to the Lahja repository.