egma-ai

homi

README

License: apache-2.0

Problem

A lot of paralinguistic and phonological questions about audio cannot be answered by looking at a transcript. This model takes in audio natively and answers those questions, e.g. "How does the speaker feel?" with the options angry / happy / sad / fear / disgust.

Just like Jev, there are three question types:

Table with columns: Type, You give, You get
Type	You give	You get
`noul`	a yes/no question	P(yes)
`choice`	a question and 2 to 255 options	a probability per option
`score`	a question and 2 to 10 ordered levels	a probability per level, and the expected level

Model

It is Qwen3-Omni-30B-A3B-Instruct with frozen weights, a decision head in place of the language-model head, 17M parameters of LoRA, and one calibration temperature. The base weights are downloaded from Qwen's repository; this repository holds only the adapter, the head and the code.

Code, evaluation and training data builders: egma-ai/homi.

Quick start

One GPU with about 70 GB free (bf16). Tested with Python 3.12, torch 2.14.1, transformers 5.18.0, peft 0.21.2.

bash

pip install torch==2.14.1 transformers==5.18.0 peft==0.21.2 accelerate safetensors soundfile scipy numpyhf download egma-ai/homi --local-dir homipython homi/predict.py clip.wav

python

import sys; sys.path.insert(0, "homi")from predict import AudioDecider decider = AudioDecider("homi")decider.ask("clip.wav", {    "angry": {"type": "noul", "instructions": "Does the speaker sound angry?"},    "emotion": {"type": "choice", "instructions": "Which emotion does the speaker express?",                "criteria": {"anger": None, "happy": None, "neutral": None, "sad": None}},    "pace": {"type": "score", "instructions": "How fast is the person speaking?",             "criteria": ["slow", "normal pace", "fast"]},})

On an angry clip from RAVDESS:

json

{"angry": {"type": "noul", "noul": 0.7024}, "emotion": {"type": "choice", "choice": "anger", "confidence": 0.9609,             "probabilities": {"anger": 0.9609, "happy": 0.0124, "neutral": 0.0154, "sad": 0.0113}}, "pace": {"type": "score", "score": 1.0611, "confidence": 0.4426,          "legend": {"0": "slow", "1": "normal pace", "2": "fast"},          "probabilities": {"0": 0.2481, "1": 0.4426, "2": 0.3093}}}

Options can carry descriptions: "criteria": {"low": "below 60 dB", "high": "60 dB or more"}.
Text context can go with the audio: decider.ask(audio, questions, text="...").
All questions about a clip run in one batched forward pass: about 75 ms for three questions on a short clip (H200).
Load it with predict.py. The adapter targets the Qwen3-Omni thinker, so AutoPeftModel will not find it.

How it works

The prompt is the audio, then the question with lettered options (A, B, C, ...). The decision head reads the hidden state at the last prompt position and gives one score per answer code; codes beyond the question's options are masked, and a softmax with the fitted temperature gives the probabilities. The head starts as the base model's own output rows for the answer-code tokens.

Results

Same audio, question and options for every model. Unanswered counts as wrong. Our cost is GPU time on one H200 ($4.54/h, batched); API costs are reported tokens at list prices (Gemini 3.8 Flash at its promotional price).

VocalGrad (7,200 questions: does pitch, speed or volume rise or fall within a clip? Chance is 50%.)

Table with columns: Model, VocalGrad, Speed, Pitch, Volume, $ per 10k questions
Model	VocalGrad	Speed	Pitch	Volume	$ per 10k questions
homi	73.9	74	61	87	0.12
Gemini 3.8 Flash	63.0	78	56	55	12.10
Qwen3-Omni-30B-A3B	50.3	50	50	50	0.11
Gemini 3.5 Flash-Lite	50.9	51	51	50	1.30
OpenAI gpt-audio	33.8	32	35	34	37.03

MMSU (4,981 questions, 47 tasks)

Table with columns: Model, MMSU, Perception, Reasoning, $ per 10k questions
Model	MMSU	Perception	Reasoning	$ per 10k questions
Gemini 3.8 Flash	85.7	82.4	89.2	14.80
homi	83.1	80.4	86.0	0.28
Qwen3-Omni-30B-A3B	72.2	60.5	84.5	0.19
Gemini 3.5 Flash-Lite	67.4	55.1	80.3	2.07
OpenAI gpt-audio	60.3	46.5	74.9	41.69

gpt-audio did not answer 34% of VocalGrad and 13% of MMSU; on the questions it did answer it scores 51.2 and 69.5. Raw answers and per-task tables are in the code repository.

Calibration (does confidence match accuracy?). "Sure" means confidence of at least 0.9.

Table with columns: Model, VocalGrad ECE, VocalGrad Brier, VocalGrad: accuracy when sure (share sure), MMSU ECE, MMSU Brier, MMSU: accuracy when sure (share sure)
Model	VocalGrad ECE	VocalGrad Brier	VocalGrad: accuracy when sure (share sure)	MMSU ECE	MMSU Brier	MMSU: accuracy when sure (share sure)
homi	0.157	0.399	82.8% (67%)	0.076	0.259	91.7% (73%)
Gemini 3.8 Flash	0.237	0.570	70.2% (37%)	0.052	0.238	93.7% (63%)
Qwen3-Omni-30B-A3B	0.337	0.740	52.5% (38%)	0.182	0.439	84.4% (73%)
Gemini 3.5 Flash-Lite	0.229	0.622	55.0% (15%)	0.133	0.512	72.1% (14%)
OpenAI gpt-audio	0.269	0.657	50.1% (8%)	0.081	0.459	84.8% (11%)

Gemini and OpenAI probabilities are the ones the models report in their answers. gpt-audio is scored only on the questions it answered. On VocalGrad, homi is the only model with a Brier score better than guessing evenly, but it is overconfident: its most confident answers (98% on average) are right 85% of the time. Full tables: eval/results/CALIBRATION.md in the code repository.

Limitations

Counting distinct speakers is weak: 19% on MMSU's speaker counting. It usually answers one too few.
Weaker than Gemini on long conversations with several people: roles, emotion between speakers, content.
Rhyming couplets (52%) and reasoning from pauses (61%) on MMSU are below the base model.
Real-vs-synthetic speech detection is not reliable. Do not use it to detect deepfakes.
Emotion was learned mostly from acted speech.
Mostly English.
One temperature for every question type. Check calibration on your own data before setting thresholds.

Training

Data: public, permissively licensed audio and text, plus synthetic audio with labels exact by construction. No customer data, and none of the datasets MMSU builds its audio from. Full list with licences: DATA.md.

LoRA rank 16 (alpha 32, dropout 0.05) on q/k/v/o of the language model's 48 layers and q/k/v of the audio encoder's 32 layers. Base weights frozen.
Soft-target cross-entropy at the last prompt position. AdamW, 64 questions per step, cosine schedule, 8 GPUs.
Two passes: 5,654 steps at learning rate 1e-4 on the core data, then 2,136 steps at 5e-5 on newer data plus a 20% replay of the core data. 498,491 training passes in total.
Text questions blend 50/50 with soft labels from perplexity-ai/pplx-decider-v1.1-27b where it is right.
Temperature T = 1.01, fitted on 16,997 calibration questions with sources weighted equally.

Intended use

Answering questions about recorded audio where a probability is more useful than free text: tone and emotion, speaking rate, pauses and turn-taking, background sounds, audio quality, what was said.

Not for: identifying or verifying who is speaking; decisions about people without human review; inferring sensitive traits from a voice; detecting lies or deepfakes; analysing people who have not consented where consent is required.

Licence

Apache-2.0 (LICENSE, NOTICE). The base model is Apache-2.0. CREMA-D: "This model was trained using CREMA-D (Cao et al., 2014), made available under the Open Database License (ODbL v1.0); contents under the Database Contents License (DbCL v1.0)." Per-clip and per-track credits: ATTRIBUTION_FSD50K.txt, ATTRIBUTION_FMA.txt, ATTRIBUTION_MUSAN.txt. One known licence exception is described at the end of DATA.md.

bibtex

@misc{homi_2026,  title        = {homi: typed questions about audio, with probabilities},  author       = {{Outlier Humans Inc.}},  year         = {2026},  howpublished = {\url{https://huggingface.co/egma-ai/homi}}}

References

Inspiration

The name is a nod to Homi J. Bhabha.
Jev (TypeSafe AI): launch post
Clef (Cloudflare): blog post, weights
pplx-decider (Perplexity): weights

Model and method

Qwen3-Omni-30B-A3B-Instruct: weights, Qwen3-Omni Technical Report
Hu et al., 2021. LoRA: Low-Rank Adaptation of Large Language Models

Benchmarks

MMSU (ICLR 2026): paper, code, data
VocalGrad: code, data
VoxParadox: paper, code
MSU-Bench: paper, code

Example clip

RAVDESS

Compared models

Gemini 3.8 Flash, Gemini 3.5 Flash-Lite: Gemini API models
OpenAI gpt-audio: model page

Training data (licences in DATA.md)

Speech: LibriTTS-R, LibriSpeech, CREMA-D (Cao et al., 2014), speechocean762, HarperValleyBank, AMI, Apollo telephony, MInDS-14, FTC robocall audio
Sounds and music: FSD50K, NSynth, FMA, OpenMIC-2018, MUSAN, OpenSLR 28 room responses
Text: CommonsenseQA, QASC, QuaRTz, JADE-Data, tasksource
Pronunciation: CMUdict, misaki
Synthetic speech and audio tools: Kokoro-82M, Piper, Parler-TTS mini v1, SpeechT5 with HiFi-GAN and CMU ARCTIC x-vectors, Chatterbox, Vocos, WORLD / pyworld, Rubber Band

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider