GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
zhifeixie
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
zhifeixie
Model Tree
Input Modalities
Output Modalities
Supported Functionality
KEEP_SILENCE / TEXT_BEGIN) and SPEAKING (emits text until TEXT_END)text
Mini-Omni3/├── model-00001-of-00004.safetensors # LM weights, sharded (≈4 GB each)├── model-00002-of-00004.safetensors├── model-00003-of-00004.safetensors├── model-00004-of-00004.safetensors├── model.safetensors.index.json # Shard index consumed by safetensors loader├── config.json # Top-level model config├── generation_config.json # Generation defaults├── model_config.yaml # GPT config consumed by Config.from_file├── hyperparameters.yaml # Training-time hyperparameters (reference)├── tokenizer.json # Tokenizer├── tokenizer_config.json├── MiniOmni3_ChunkwisedEncoder.pth # Audio encoder weights (Qwen2.5-Omni audio tower)└── qwen25OmniConfig/ # Audio-encoder config (nested: thinker_config.audio_config)Mini-Omni3 is intended for streaming conversational agents that need to react to audio as it arrives — for example, voice assistants that may interject mid-utterance, alarms that respond to ambient sound, or low-latency dialogue systems where waiting for a full utterance before replying is too slow. The model is not a transcription system; it produces a conversational reply (or silence) rather than a verbatim transcript.
bash
git clone https://github.com/xzf-thu/Mini-Omni3.git # TODO: confirm repo URLcd Mini-Omni3conda create -n mini-omni3 python=3.10 -yconda activate mini-omni3pip install -r requirements.txtFrom the Mini-Omni3 project root, pull the weights into checkpoints/:
python
from huggingface_hub import snapshot_download snapshot_download(repo_id="zhifeixie/Mini-Omni3", local_dir="checkpoints")snapshot_download is the recommended path — it pulls every file, resumes on interruption, and is the only way the download counter on this page advances. Please avoid git clone of the HF repo or the web "Download" button if you want your run reflected in the stats.
python
from src.miniomni3.generate.run import run_inference run_inference( checkpoint_dir="checkpoints", audio_paths=["/path/to/audio.wav"], # offline mode: one round per path device="cuda:0", # or "mps" / "cpu")For interactive use, omit audio_paths and run_inference will prompt for an audio path each round:
python
run_inference(checkpoint_dir="checkpoints", rounds=5, device="cuda:0")A single session looks like:
text
[system prompt tokens] ┌─── LISTENING ───┐ │ AUDIO_BEGIN PAD*10 ASSISTANT → KEEP_SILENCE (keep listening) │ AUDIO_BEGIN PAD*10 ASSISTANT → TEXT_BEGIN EMOTION (start replying) └─────────────────┘ ┌─── SPEAKING ────┐ │ … text tokens … TEXT_END (reply finished) └─────────────────┘ ┌─── LISTENING ───┐ (next audio chunk) …The model is trained to emit at most one TEXT_BEGIN per audio chunk. Each assistant turn begins with TEXT_BEGIN, followed by an emotion token, the reply tokens, and TEXT_END. Turns starting with KEEP_SILENCE indicate the model chose not to respond to that chunk.
whisper.load_audio and padded to 0.4-second boundaries before encoding.bibtex
@misc{xie_miniomni3, title = {Mini-Omni3: Streaming Audio-In, Text-Out Conversational Modeling}, author = {Zhifei Xie and collaborators}, year = {2026}, note = {Preprint in preparation}}Mini-Omni3 builds on the Qwen2.5-Omni audio encoder. We thank the Qwen team and the maintainers of OpenAI Whisper for the audio-loading utilities used in this project.
Mini-Omni3/├── model-00001-of-00004.safetensors # LM weights, sharded (≈4 GB each)├── model-00002-of-00004.safetensors├── model-00003-of-00004.safetensors├── model-00004-of-00004.safetensors├── model.safetensors.index.json # Shard index consumed by safetensors loader├── config.json # Top-level model config├── generation_config.json # Generation defaults├── model_config.yaml # GPT config consumed by Config.from_file├── hyperparameters.yaml # Training-time hyperparameters (reference)├── tokenizer.json # Tokenizer├── tokenizer_config.json├── MiniOmni3_ChunkwisedEncoder.pth # Audio encoder weights (Qwen2.5-Omni audio tower)└── qwen25OmniConfig/ # Audio-encoder config (nested: thinker_config.audio_config)git clone https://github.com/xzf-thu/Mini-Omni3.git # TODO: confirm repo URLcd Mini-Omni3conda create -n mini-omni3 python=3.10 -yconda activate mini-omni3pip install -r requirements.txtfrom huggingface_hub import snapshot_download snapshot_download(repo_id="zhifeixie/Mini-Omni3", local_dir="checkpoints")from src.miniomni3.generate.run import run_inference run_inference( checkpoint_dir="checkpoints", audio_paths=["/path/to/audio.wav"], # offline mode: one round per path device="cuda:0", # or "mps" / "cpu")run_inference(checkpoint_dir="checkpoints", rounds=5, device="cuda:0")[system prompt tokens] ┌─── LISTENING ───┐ │ AUDIO_BEGIN PAD*10 ASSISTANT → KEEP_SILENCE (keep listening) │ AUDIO_BEGIN PAD*10 ASSISTANT → TEXT_BEGIN EMOTION (start replying) └─────────────────┘ ┌─── SPEAKING ────┐ │ … text tokens … TEXT_END (reply finished) └─────────────────┘ ┌─── LISTENING ───┐ (next audio chunk) …@misc{xie_miniomni3, title = {Mini-Omni3: Streaming Audio-In, Text-Out Conversational Modeling}, author = {Zhifei Xie and collaborators}, year = {2026}, note = {Preprint in preparation}}