TalTechNLP

Voxtral-Mini-3B-2507-estonian

README

License: apache-2.0

Model details

Table with columns: Property, Value
Property	Value
Base model	`mistralai/Voxtral-Mini-3B-2507`
Architecture	`VoxtralForConditionalGeneration`
Primary language	Estonian (`et/est`)
Additional output language	English (`en`)
Fine-tuning method	Full-parameter supervised fine-tuning
Selected checkpoint	4,500 steps
Weight precision	bfloat16
License	Apache 2.0, following the base model

Intended uses

The model is intended for research and development involving Estonian speech, including:

verbatim automatic speech recognition;
readable subtitle and parliamentary stenogram generation;
summaries and news stories based on broadcast audio;
answering questions about spoken content; and
translation of Estonian speech into English.

The output style depends on the instruction. Prompts should state the desired task and format explicitly. This is not a certified system for legal, medical, safety-critical, or high-stakes transcription.

Usage with Transformers

The following example uses the selected local checkpoint for Estonian transcription:

python
import torch
from transformers import AutoProcessor, VoxtralForConditionalGeneration

model_id = "exp/models/voxtral-mini-3b-2507-full/1c/checkpoint-4500"
device = "cuda"

processor = AutoProcessor.from_pretrained(model_id)
model = VoxtralForConditionalGeneration.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="sdpa",
    low_cpu_mem_usage=True,
).to(device)
model.eval()

inputs = processor.apply_transcription_request(
    language="et",
    audio="example.wav",
    model_id=model_id,
    return_tensors="pt",
)
inputs = {
    key: value.to(device=device, dtype=torch.bfloat16)
    if value.is_floating_point()
    else value.to(device)
    for key, value in inputs.items()
}

with torch.inference_mode():
    output_ids = model.generate(**inputs, do_sample=False, max_new_tokens=8192)

prompt_length = inputs["input_ids"].shape[1]
text = processor.batch_decode(
    output_ids[:, prompt_length:], skip_special_tokens=True
)[0]
print(text.strip())

Use 16 kHz mono audio where possible. For long recordings, segmenting at natural speech boundaries is recommended.

Training data

The training data was combined from seven task datasets.

Table with columns: Task, Input, Retained, Filtered out
Task	Input	Retained	Filtered out
Verbatim Estonian ASR	13,434	12,972	462
Edited broadcast subtitles	5,639	5,412	227
Estonian news stories from audio	50,000	49,998	2
English news stories from Estonian audio	10,000	9,999	1

Of the 7,035 filtered examples, 5,500 exceeded the task-specific audio-duration limit and 1,535 exceeded the maximum combined audio/text sequence length. Transcription examples were limited to 30 minutes and audio-understanding examples to 40 minutes. The final prepared manifest used a maximum sequence length of 28,000 tokens.

Combined sequence lengths in the retained manifest were:

Table with columns: Statistic, Tokens
Statistic	Tokens
Minimum	419
Median	6,845
90th percentile	19,163
95th percentile	19,734
99th percentile	24,925
Maximum	27,999

Some instructions and answers in trainining data and the English news-story targets were generated with google/gemini-3.1-flash-lite. Consequently, the training targets may contain errors or stylistic biases inherited from that model.

The source collections include ERR broadcasts and subtitles, an Estonian ASR manifest, ERR news video/audio and article text, Päevakaja programmes, Riigikogu recordings and stenograms, and TalTechNLP/qa_broadcast_conv_et. Users are responsible for checking the terms, copyright, privacy constraints, and permitted uses of the underlying data for their application.

Training procedure

Training used four NVIDIA A100-SXM4 80 GB GPUs. The per-device batch size was 1 with 8 gradient-accumulation steps, giving an effective global batch size of 32. Both the audio encoder and aligner were unfrozen.

Table with columns: Hyperparameter, Value
Hyperparameter	Value
Maximum training steps	6,000
Selected checkpoint step	4,500
Effective global batch size	32
Optimizer	Fused AdamW
Learning rate	`5e-5`
Adam betas	`(0.9, 0.95)`
Adam epsilon	`1e-8`

The run was resumed from checkpoints during development, and the configured maximum sequence length was reduced from 32,768 to 30,000 and finally 28,000. The data statistics above describe the final prepared 28,000-token manifest; they should not be interpreted as an exact record of the manifest state at every earlier resumed stage.

Evaluation

Checkpoints were ranked by word error rate on an internal held-out Estonian ASR set. Generation used greedy decoding (temperature=0) with up to 8,192 new tokens.

Table with columns: Checkpoint, WER, Reference words
Checkpoint	WER	Reference words
4,500	5.05%	9,085

The validation set contains only 10 recordings. These figures are useful for checkpoint selection but should not be treated as a broad estimate of Estonian ASR quality or as directly comparable with public benchmark results.

Toolkits and environment

The recorded environment for the final training stage included:

Table with columns: Toolkit, Version or role
Toolkit	Version or role
ms-swift	4.4.2; supervised fine-tuning and evaluation loop
Transformers	4.57.6; Voxtral model and processor
PyTorch	2.10.0
Datasets	3.6.0
Accelerate	1.12.0
Mistral Common	1.11.7
kaldiio	2.18.1; reading FLAC `ark` audio
SoundFile	0.13.1

A local ms-swift plugin supplied Voxtral dataset integration and support for the repository's ark:<path>:<offset> audio references.

Limitations

The model can hallucinate, omit, repeat, or incorrectly normalize speech.
Performance may degrade for dialects, code-switching, overlapping speakers, noise, distant microphones, music, or domains underrepresented in training.
The English outputs were trained on machine-generated references and may inherit translation errors.
Generated summaries and answers are not guaranteed to be factually faithful to the recording.
Training data may contain personal names, opinions, and other information present in public broadcasts or parliamentary speech. Outputs should be reviewed before publication or consequential use.

Acknowledgements

The model builds on Mistral AI's Voxtral Mini 3B 2507. See the upstream model card for architecture details, original capabilities, usage requirements, and base-model limitations.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

TalTechNLP

Model Tree

Base

mistralai/Voxtral-Mini-3B-2507

Fine-tuned

this model

Input Modalities

TextAudio

Output Modalities

Text

Supported Functionality