GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
adventists-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
adventists-ai
Model Tree
openai/whisper-small
Input Modalities
Output Modalities
Supported Functionality
bash
pip install "duplexjev[speech]>=0.2.2"python
from duplexjev import Decider, Question d = Decider.from_pretrained("adventists-ai/DuplexJev-B-Whisper-small-Falcon-H1-1.5B") d.decide("utterance_en.wav", [ Question("turn", "Has the user finished speaking?", ["finished", "not finished"], lang="en"), Question("intent", "What does the user want?", ["climate", "media", "navigation", "phone", "chit-chat"], lang="en"),], lang="en")d.decide("utterance_zh.wav", [ # Chinese speech: Chinese question and options Question("turn", "用户说完了吗？", ["说完了", "还没说完"], lang="zh"), Question("intent", "用户想做什么？", ["空调", "媒体", "导航", "电话", "闲聊"], lang="zh"),], lang="zh")Falcon-H1 is a hybrid attention + Mamba model. Recurrent layers carry state along the sequence, so duplexjev (≥ 0.2.2) turns off prefix packing for it automatically (mode="batch": one row per question). For GPU speed install the Mamba kernels: pip install mamba-ssm causal-conv1d (they must be built for your torch/CUDA); without them transformers falls back to a slow reference path.
| Evaluation (%) | paper protocol | duplexjev 0.2.2 defaults |
|---|---|---|
| qa100 (spoken QA) | 50 | 49 |
| qa100, Falcon-H1-1.5B reading the transcript | 73 | – |
| ZJU-ML (spoken QA, real + TTS) | 34 | – |
| Easy-Turn (4-way turn state, 800 clips) | 31.9 | – |
| Gender, 800 real utterances | 53.1 | 54.1 |
| Emotion, 4-way, 800 utterances | 26.9 | 26.8 |
Leaderboard scores (see the GitHub README): main language (mean of qa100, ZJU-ML, Easy-Turn) 38.6, paralinguistics (mean of gender, emotion) 40.0.
Paper protocol: the evaluation prompts of the paper (single option order, one H200). duplexjev defaults: the package's
own prompts with lang matched to the clip. Spoken QA with a small LLM is bounded by the LLM itself (see the
transcript-reading row); use these connectors for short decisions and speaker cues rather than open knowledge questions.
Latency of one decision event (10 questions, one 4.5 s clip, duplexjev 0.2.2, plain PyTorch): 85.3 s on 8 CPU threads (Xeon Platinum 8558, fp32, not quantized); about 187 ms on one H200 (bf16, measured while the GPU was shared with a training job). Falcon-H1 is not CPU-friendly: without Mamba CPU kernels transformers uses a slow reference path, and each question needs its own row.
Frozen encoder and LLM; only the connector is trained: R1 and R2 transcript distillation on two disjoint 0.5 M-utterance packs of the Ultravox v0.6 mixture (32k steps each, global batch 16). Code and scripts: https://github.com/adventists-ai/duplexjev.
Apache-2.0 for the connector weights. Some training corpora (e.g. WenetSpeech, CoVoST 2) are licensed for non-commercial use only; check them before commercial use. The frozen models keep their own licences: Falcon-H1-1.5B (TII Falcon License 2.0) and the encoder (Apache-2.0).
bibtex
@inproceedings{jin2027duplexjev, title = {Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen {LLM} Hear Beyond the Transcript}, author = {Jin, Jie and Ma, Ziyin and Yin, Min and Chen, Jinyu and Song, Haigang and Pang, Zhikun and Zhang, Xiaowen}, booktitle = {Submitted to IEEE ICASSP}, year = {2027}}Built by Adventists.ai. Claude (Anthropic) assisted with code.
from duplexjev import Decider, Question d = Decider.from_pretrained("adventists-ai/DuplexJev-B-Whisper-small-Falcon-H1-1.5B") d.decide("utterance_en.wav", [ Question("turn", "Has the user finished speaking?", ["finished", "not finished"], lang="en"), Question("intent", "What does the user want?", ["climate", "media", "navigation", "phone", "chit-chat"], lang="en"),], lang="en")d.decide("utterance_zh.wav", [ # Chinese speech: Chinese question and options Question("turn", "用户说完了吗？", ["说完了", "还没说完"], lang="zh"), Question("intent", "用户想做什么？", ["空调", "媒体", "导航", "电话", "闲聊"], lang="zh"),], lang="zh")@inproceedings{jin2027duplexjev, title = {Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen {LLM} Hear Beyond the Transcript}, author = {Jin, Jie and Ma, Ziyin and Yin, Min and Chen, Jinyu and Song, Haigang and Pang, Zhikun and Zhang, Xiaowen}, booktitle = {Submitted to IEEE ICASSP}, year = {2027}}