danielrosehill

Whisper-Hebrish-Small-ACFT

Install on your phone (FUTO Keyboard)

On the phone, download ggml-whisper-hebrish-small-en-acft-q8_0.bin (264 MB).
Import it in either of these ways:
- tap the downloaded file and choose FUTO Keyboard under "Open with", or
- open FUTO Keyboard Settings → Languages & Models → Import resource file and pick the .bin.
The importer shows "Import - Voice Input Model" and asks you to choose a language. Pick English (the language you dictate in). The model replaces English's voice input model. The app does not ask whether the model is English-only or multilingual; whisper.cpp works that out from the file.
Use voice input as usual. The language's entry under Languages & Models now shows the voice model as (Imported).
To undo, open the same entry, tap the voice input model, and choose Revert to Default.

Optional, not tested with this model: Personal Dictionary for Voice (in voice input settings) sends your personal dictionary words to Whisper as a (Glossary: ...) prompt. Adding the Hebrew words you use most may help further.

These steps come from FUTO Keyboard's source (ImportResourceActivity.kt, commit 70a5d390, 2026-09-23), not from a screen recording. The importer identifies the file as a voice model from its first four bytes (whisper.cpp's ggml magic) and then asks for a language. The file also went through FUTO's own bundled whisper.cpp, built on a desktop, with the keyboard's exact inference settings. That check produced the "FUTO runtime" rows below.

Results

The test sets are the same as Whisper Hebrish v2's, so the numbers are directly comparable:

test_cs: 155 English sentences with Hebrew words. Its Hebrew terms are disjoint from training and it was never trained on.
held-term recall: the share of those test-only Hebrew terms that come out in the canonical spelling.
own EN: 33 of the speaker's plain English sentences.
Libri: 200 LibriSpeech test-clean clips.

WER is normalised (case and punctuation removed); lower is better. "FUTO ctx" is FUTO's runtime context, max(160, min(1500, ceil(samples/320) + 32)) encoder positions. "Full" is the normal 30-second window.

Table with columns: Model, Context / decoding, test_cs WER, held-term recall, own EN WER, Libri WER
Model	Context / decoding	test_cs WER	held-term recall	own EN WER	Libri WER
`openai/whisper-small.en` (stock)	full, greedy	0.117	0.26	0.044	0.024
`openai/whisper-small.en` (stock)	FUTO ctx, greedy	0.941	0.23	0.978	0.241

On the phone, this model halves FUTO's stock English-244 error rate on code-switched speech (0.118 → 0.058). It gets about twice as many unseen Hebrew terms right (0.26 → 0.50), and plain English improves too (0.041 → 0.028). It comes within about 0.006 WER of the large v2 model at a quarter of the parameters. ACFT costs nothing at full context, and under FUTO's context it brings the fine-tune from 1.23 WER (unusable) to 0.062.

Known weakness: dictations longer than 30 seconds. The test_long set (stitched 30 to 60 s clips, run through the Transformers chunked pipeline) scores 0.21 to 0.27 WER, against 0.09 for v2. The small model handles a 30-second boundary worse. Typical keyboard dictation is shorter than that.

Per-utterance hypotheses for every row are in evals/. The scoring code is in the training repository.

Training

Fine-tune. Full fine-tune of openai/whisper-small.en on 1,929 clips (215 minutes) of the speaker's read speech:
- 70% code-switching sentences, 20% his own plain English, and 10% public English replay to avoid forgetting.
- The target text is cased and punctuated, with Hebrew in canonical romanization.
- 10 epochs, lr 1.25e-5 (linear, 10% warmup), effective batch 32, bf16 autocast.
- The best checkpoint was chosen on validation WER (epoch 8, 0.033).
- The data, mix and loop are identical to Whisper Hebrish v2; only the base and the learning rate differ.
ACFT. FUTO's procedure (futo-org/whisper-acft finetune.ipynb at commit f0700d0), with FUTO's settings unchanged:
- Adam, lr 1e-6, batch 1, 8 epochs.

training/ holds the run configs, the ACFT log (acft-result.json) and the conversion check (ggml-verify.json).

Use outside FUTO

bash

# whisper.cpp, with the same dynamic context FUTO uses (here for a 6 s clip: 6*50+32 = 332)whisper-cli -m ggml-whisper-hebrish-small-en-acft-q8_0.bin -f clip.wav -bs 5 -ac 332

python

from transformers import pipelineasr = pipeline("automatic-speech-recognition", model="danielrosehill/Whisper-Hebrish-Small-ACFT")print(asr("clip.wav")["text"])   # English-only model: do not pass language/task

License

MIT. The base model openai/whisper-small.en is MIT. The ACFT method is from FUTO's whisper-acft repository (MIT).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

danielrosehill

Model Tree

Base

openai/whisper-small.en

Fine-tuned

this model

Input Modalities

Audio

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Install on your phone (FUTO Keyboard)

On the phone, download ggml-whisper-hebrish-small-en-acft-q8_0.bin (264 MB).
Import it in either of these ways:
- tap the downloaded file and choose FUTO Keyboard under "Open with", or
- open FUTO Keyboard Settings → Languages & Models → Import resource file and pick the .bin.
The importer shows "Import - Voice Input Model" and asks you to choose a language. Pick English (the language you dictate in). The model replaces English's voice input model. The app does not ask whether the model is English-only or multilingual; whisper.cpp works that out from the file.
Use voice input as usual. The language's entry under Languages & Models now shows the voice model as (Imported).
To undo, open the same entry, tap the voice input model, and choose Revert to Default.

Results

The test sets are the same as Whisper Hebrish v2's, so the numbers are directly comparable:

test_cs: 155 English sentences with Hebrew words. Its Hebrew terms are disjoint from training and it was never trained on.
held-term recall: the share of those test-only Hebrew terms that come out in the canonical spelling.
own EN: 33 of the speaker's plain English sentences.
Libri: 200 LibriSpeech test-clean clips.

Table with columns: Model, Context / decoding, test_cs WER, held-term recall, own EN WER, Libri WER
Model	Context / decoding	test_cs WER	held-term recall	own EN WER	Libri WER
`openai/whisper-small.en` (stock)	full, greedy	0.117	0.26	0.044	0.024
`openai/whisper-small.en` (stock)	FUTO ctx, greedy	0.941	0.23	0.978	0.241

Per-utterance hypotheses for every row are in evals/. The scoring code is in the training repository.

Training

Fine-tune. Full fine-tune of openai/whisper-small.en on 1,929 clips (215 minutes) of the speaker's read speech:
- 70% code-switching sentences, 20% his own plain English, and 10% public English replay to avoid forgetting.
- The target text is cased and punctuated, with Hebrew in canonical romanization.
- 10 epochs, lr 1.25e-5 (linear, 10% warmup), effective batch 32, bf16 autocast.
- The best checkpoint was chosen on validation WER (epoch 8, 0.033).
- The data, mix and loop are identical to Whisper Hebrish v2; only the base and the learning rate differ.
ACFT. FUTO's procedure (futo-org/whisper-acft finetune.ipynb at commit f0700d0), with FUTO's settings unchanged:
- Adam, lr 1e-6, batch 1, 8 epochs.

training/ holds the run configs, the ACFT log (acft-result.json) and the conversion check (ggml-verify.json).

Use outside FUTO

bash

# whisper.cpp, with the same dynamic context FUTO uses (here for a 6 s clip: 6*50+32 = 332)whisper-cli -m ggml-whisper-hebrish-small-en-acft-q8_0.bin -f clip.wav -bs 5 -ac 332

python

from transformers import pipelineasr = pipeline("automatic-speech-recognition", model="danielrosehill/Whisper-Hebrish-Small-ACFT")print(asr("clip.wav")["text"])   # English-only model: do not pass language/task

License

MIT. The base model openai/whisper-small.en is MIT. The ACFT method is from FUTO's whisper-acft repository (MIT).

Whisper-Hebrish-Small-ACFT

README

Install on your phone (FUTO Keyboard)

Results

Training

Use outside FUTO

License

Explore FriendliAI today

README

Install on your phone (FUTO Keyboard)

Results

Training

Use outside FUTO

License