GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
itayinbar
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
itayinbar
Model Tree
itayinbar/whisper-he-100m
Input Modalities
Output Modalities
Supported Functionality
Scored with a harness that reproduces the ivrit.ai Hebrew leaderboard on 40 of 40 published model and dataset pairs.
| Benchmark | this model, 52M | whisper-he-100m, 94M | whisper-base-he, 50M | ivrit.ai, 809M |
|---|---|---|---|---|
ivrit-ai/eval-d1 | 10.12% | 12.99% | 14.87% | 5.5% |
ivrit-ai/eval-whatsapp | 18.05% | 21.67% | 25.77% | 6.1% |
imvladikon/hebrew_speech_kan | 14.21% | 16.62% | 18.47% | 8.10% |
google/fleurs he | 31.10% | 32.19% | 33.74% | 18.72% |
It is also the fastest of the three: 506 to 635 times realtime on one laptop GPU against 276 to 377 for whisper-he-100m. On a phone the gap should be wider, since the decoder runs once per output token and this one has 2 layers instead of 12.
Architecture. Distil-Whisper's shape: keep the whole encoder, keep two decoder layers. Starting from whisper-he-100m (12 encoder and 12 decoder layers), the first and last decoder layers are kept and the other ten removed, taking it from 94.4M to 52.3M parameters. The kept layers are copied as trained, so the student begins with a decoder that already knows the vocabulary and the cross-attention to this encoder.
Data. The earlier models learned mostly from Knesset plenum transcripts,
2,757 of their 3,113 hours, which are aligned protocols rather than verbatim
speech. This one replaces them with teacher labels: ivrit.ai's
whisper-large-v3-turbo transcribed whole episodes of ivrit-ai/audio-v2
(podcasts and interviews, no source over 10% of the total), cut into windows of
up to 28 seconds and filtered on the decoder's own confidence. The filter cutoffs
were checked against human transcripts first: the teacher scores 6.85% WER on
crowd-transcribe, and its confidence barely predicts its errors, so Whisper's
default cutoffs were kept rather than throwing away audio for nothing.
Mix. 75% teacher labels, 15% ivrit-ai/crowd-transcribe-v5, 4%
ivrit-ai/crowd-recital, 3% google/fleurs he and 3%
imvladikon/hebrew_speech_kan, as shares of audio heard. No Knesset.
The tokenizer is unchanged from the earlier models: an 8,192-token Hebrew byte-level BPE, 1.76 tokens per word against Whisper's 3.17.
python
from transformers import AutoModelForSpeechSeq2Seq, AutoTokenizer, AutoFeatureExtractorfrom transformers.generation.utils import GenerationMixin model = AutoModelForSpeechSeq2Seq.from_pretrained("itayinbar/whisper-he-12x2")tokenizer = AutoTokenizer.from_pretrained("itayinbar/whisper-he-12x2")features = AutoFeatureExtractor.from_pretrained("itayinbar/whisper-he-12x2") inputs = features(audio_16khz, sampling_rate=16000, return_tensors="pt", padding="max_length")# Monolingual: no language token, no task token, no timestamps.ids = GenerationMixin.generate(model, **inputs, max_new_tokens=200)print(tokenizer.decode(ids[0], skip_special_tokens=True))Three things differ from stock Whisper:
generate. There are no language or task tokens.A browser downloads 130.6 MB, the fp16 encoder with the fp32 decoder. Only those weights are shipped: an fp16 decoder fails to load in ONNX Runtime and an int8 encoder loads but changes the transcript. Every combination offered here was checked by transcribing with it and comparing against the fp32 output.
2,069.8 hours of teacher-labelled ivrit-ai/audio-v2 (a further 15.1 hours held
out to choose the checkpoint) plus 356.1 hours of human transcripts.
Schedule-free AdamW, learning rate 1e-4, effective batch 16, bf16, on a single
RTX 5070 Laptop GPU. The checkpoint is from step 84,000, where error on the
held-out conversational audio was lowest (16.01% against the teacher's own
transcripts, where whisper-he-100m scores 20.92%); later passes over the same
audio only lowered the training loss.
Weights are Apache-2.0, following openai/whisper-base and
ivrit-ai/whisper-large-v3-turbo.
Training audio and transcripts from ivrit.ai under the
ivrit.ai licence, which permits training
models including commercially and requires attribution. FLEURS is CC-BY-4.0.
imvladikon/hebrew_speech_kan declares no licence on the Hub; it contributed 3%
of the audio heard and is named so anyone relying on provenance can judge it.
Code and method: github.com/itayinbarr/Hebrew-small-asr
from transformers import AutoModelForSpeechSeq2Seq, AutoTokenizer, AutoFeatureExtractorfrom transformers.generation.utils import GenerationMixin model = AutoModelForSpeechSeq2Seq.from_pretrained("itayinbar/whisper-he-12x2")tokenizer = AutoTokenizer.from_pretrained("itayinbar/whisper-he-12x2")features = AutoFeatureExtractor.from_pretrained("itayinbar/whisper-he-12x2") inputs = features(audio_16khz, sampling_rate=16000, return_tensors="pt", padding="max_length")# Monolingual: no language token, no task token, no timestamps.ids = GenerationMixin.generate(model, **inputs, max_new_tokens=200)print(tokenizer.decode(ids[0], skip_special_tokens=True))