diarizers-community

DiarizationLM-Gemma-4-E4B-v1

README

License: apache-2.0

Overview

DiarizationLM is a Large Language Model framework designed to post-process, correct, and optimize automatic speech recognition (ASR) and speaker diarization outputs.

diarizers-community/DiarizationLM-Gemma-4-E4B-v1 is built on Google's Gemma 4 E4B (4B dense parameters) foundation model and fine-tuned with Locality-Preserving Oracle Supervision across all 4 canonical speaker diarization benchmark corpora:

Fisher English (2-speaker conversational telephone speech)
Callhome American English (2–5 speaker informal telephone conversations)
ICSI Meeting Corpus (3–9 speaker academic research meetings)
AMI Meeting Corpus (4-speaker tabletop meetings)

Foundation model: google/gemma-4-E4B
End-to-end open-source ASR + Diarization + DiarizationLM pipeline (diarizers): https://github.com/diarizers-community/diarizers (pip install diarizers)
Core DiarizationLM library & scripts (diarizationlm): https://github.com/google/speaker-id/tree/master/DiarizationLM (pip install diarizationlm)

Unlike earlier models trained exclusively on 2-speaker telephone data (such as google/DiarizationLM-8b-Fisher-v2) or naive multi-domain SFT (which suffers from long-monologue speaker identity drift in 4+ speaker meetings), diarizers-community/DiarizationLM-Gemma-4-E4B-v1 is trained on locality-preserving oracle targets that teach the model to correct lexical turn boundaries and backchannels (1 ≤ L ≤ 5 words) while preserving acoustic speaker anchors across long monologues (L ≥ 6 words). As a result, it achieves statistically significant (p < 0.0001) WDER and cpWER improvements simultaneously across all four benchmarks under both Google's USM + Turn-to-Diarize frontend and the open-source diarizers (v1.0.0) frontend, despite having half the parameter count (4B vs. 8B).

Training Configuration

Base Architecture: Gemma 4 E4B (42 layers, hidden size 2560, hybrid 5:1 sliding-window and global attention, 262,144 vocabulary size)
LoRA Adapter: Rank r = 256 applied to all attention (q_proj, k_proj, v_proj, o_proj), MLP (gate_proj, up_proj, down_proj), and per-layer input (per_layer_input_gate, per_layer_projection, per_layer_model_projection) linear projections, merged into 16-bit (bfloat16) base weights and serialized to 4-bit GGUF (Q4_K_M)
Training Objective: Completion-only cross-entropy loss (<prompt> --> <completion> [eod])
Training Data: 51,063 Fisher + 20,762 Multi-Corpus (Callhome, ICSI, AMI) locality-preserving prompt-completion pairs
Optimization: 10,000 steps, global batch size 8, AdamW (beta1 = 0.9, beta2 = 0.99), peak learning rate 1.5e-4 with 500-step linear warmup and cosine decay on 8 Google Cloud TPU v5p chips
Prompt Segmentation Length: 4,000 characters (maximal sequence length 2,560 tokens)

Model Files Included

model.safetensors: Merged 16-bit (bfloat16) Hugging Face transformers weights (~16.0 GB)
DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf: (Recommended GGUF) Serialized 4-bit K-quant Medium (Q4_K_M) GGUF model (~5.30 GB) for llama.cpp / Ollama / llama-cpp-python (uses 256-element super-blocks in Q4_K with sensitive attn_v / ffn_down and embedding matrices retained in 6-bit Q6_K)
config.json, generation_config.json, tokenizer.json, tokenizer_config.json, processor_config.json, chat_template.jinja, special_tokens_map.json: Tokenizer and model configuration files

Benchmark Performance

1. USM + Turn-to-Diarize Frontend (with 95% Bootstrap Confidence Intervals)

All metrics below are micro-averaged across the full evaluation sets using the USM + turn-to-diarize baseline and scored via Hungarian-matching dynamic programming (diarizationlm.compute_metrics_on_json_dict). Ranges in brackets indicate 95% non-parametric bootstrap confidence intervals (B = 10,000 conversation-level resamples):

Table with columns: Benchmark, Test Split, WER (%), System, WDER (%) [95% CI], cpWER (%) [95% CI], SpkCntMAE [95% CI], Paired ΔWDER vs. Baseline (p-value)
Benchmark	Test Split	WER (%)	System	WDER (%) [95% CI]	cpWER (%) [95% CI]	SpkCntMAE [95% CI]	Paired ΔWDER vs. Baseline (p-value)
Fisher	TEST FULL (172 sessions)	15.37	Baseline (USM + Turn-to-Diarize)`DiarizationLM-8b-Fisher-v2` (Llama 3 8B)`DiarizationLM-Gemma-4-E4B-v1` (4B)	5.32 [4.93, 5.74]3.282.99 [2.65, 3.37]	20.88 [19.97, 21.88]18.3717.62 [16.76, 18.56]	0.215 [0.151, 0.291]---0.093 [0.047, 0.145]	-------2.33% [-2.51, -2.16] (p < 0.0001)
Callhome	TEST FULL (20 calls)	15.22	Baseline (USM + Turn-to-Diarize)`DiarizationLM-8b-Fisher-v2` (Llama 3 8B)`DiarizationLM-Gemma-4-E4B-v1` (4B)	7.74 [6.07, 9.65]6.664.92 [3.46, 6.75]	24.31 [21.27, 27.43]23.5720.69 [17.98, 23.66]	0.050 [0.000, 0.150]---0.000 [0.000, 0.000]	-------2.82% [-3.48, -2.14] (p < 0.0001)
ICSI	TEST FULL (3 meetings)	29.42	Baseline (USM + Turn-to-Diarize)`DiarizationLM-Gemma-4-E4B-v1` (4B)	14.70 [11.65, 20.29]14.10 [10.77, 19.94]	43.90 [39.10, 51.67]43.32 [38.38, 51.20]	0.333 [0.000, 1.000]0.333 [0.000, 1.000]	----0.60% [-0.88, -0.35] (p < 0.0001)
AMI	TEST WORD FULL (16 meetings)	24.33	Baseline (USM + Turn-to-Diarize)`DiarizationLM-Gemma-4-E4B-v1` (4B)	15.68 [10.64, 21.11]14.89 [9.80, 20.38]	40.32 [32.43, 48.00]39.57 [31.57, 47.29]	0.500 [0.188, 0.812]0.438 [0.188, 0.750]	----0.79% [-1.00, -0.60] (p < 0.0001)

2. Open-Source Frontend (`diarizers` `v1.0.0` + `DiarizationLM-Gemma-4-E4B-v1`)

Below are the full-set evaluation results across the exact same 211 test recordings (172 Fisher, 20 CALLHOME, 3 ICSI, 16 AMI) when diarizers-community/DiarizationLM-Gemma-4-E4B-v1 is paired with the 100% open-source diarizers (v1.0.0) frontend (openai/whisper-large-v3-turbo ASR + pyannote/segmentation-3.0 neural powerset segmentation + speechbrain/spkrec-ecapa-voxceleb speaker embeddings + Symmetric Normalized Laplacian SpectralClusterer):

Table with columns: Benchmark (Full Test Split), Utterances, Pipeline / System, WER (%), WDER (%), cpWER (%), SpkCntMAE
Benchmark (Full Test Split)	Utterances	Pipeline / System	WER (%)	WDER (%)	cpWER (%)	SpkCntMAE
Fisher English	172	`diarizers` v1.0.0 Open Frontend (`Whisper-large-v3-turbo` + `pyannote/seg-3.0` + `ECAPA-TDNN` + `SpectralClusterer`, Fixed `K=2`)	20.80	4.35	24.78	0.000
Fisher English	172	`diarizers` v1.0.0 + `DiarizationLM-Gemma-4-E4B-v1` (`LG-TPST`)	20.80	2.59 (Raw-TPST: 2.65)	21.91 (Raw-TPST: 21.99)	0.000
CALLHOME English	20	`diarizers` v1.0.0 Open Frontend (Auto Speaker Count `[2..6]`, `eigen_threshold=0.45`)	21.25	9.42	32.42	0.050
CALLHOME English	20	`diarizers` v1.0.0 + `DiarizationLM-Gemma-4-E4B-v1`	21.25	4.89 (LG-TPST: 5.98)	25.91 (LG-TPST: 27.69)	0.050
ICSI Meeting Corpus	3	`diarizers` v1.0.0 Open Frontend (Auto Speaker Count `[2..10]`, `eigen_threshold=0.45`)	35.29	23.48	48.97	3.333
ICSI Meeting Corpus	3	`diarizers` v1.0.0 + `DiarizationLM-Gemma-4-E4B-v1` (`LG-TPST`)	35.29	23.10 (Raw-TPST: 23.11)	48.62 (Raw-TPST: 48.63)	3.333
AMI (`Mix-Headset`)	16	`diarizers` v1.0.0 Open Frontend (`AMI_MEETING_PRESET`, Auto Speaker Count `[3..5]`, `eigen_threshold=0.45`)	29.14	20.68	46.07	0.688
AMI (`Mix-Headset`)	16	`diarizers` v1.0.0 + `DiarizationLM-Gemma-4-E4B-v1`	29.14	20.13 (LG-TPST: 20.14)	45.53 (LG-TPST: 45.56)	0.688

Note: Post-LLM evaluation numbers above were computed with the full-precision (bfloat16) DiarizationLM-Gemma-4-E4B-v1 checkpoint using 6,000-character prompt chunks. For CPU deployment, this repository also provides DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf.

Usage

1. End-to-End Audio Pipeline (`diarizers`)

To transcribe, diarize, and post-process raw .wav / .mp3 audio end-to-end on CPU or GPU using the diarizers library:

bash

pip install "diarizers[audio,llm]"

python

import diarizers # CPU mode using the 4-bit Q4_K_M GGUF model (or pass llm_backend="transformers", device="cuda:0" on GPU):pipe = diarizers.DiarizersPipeline.from_pretrained(    asr_model="openai/whisper-large-v3-turbo",    diarizer_preset=diarizers.FISHER_TELEPHONY_PRESET,    llm_model_or_path="diarizers-community/DiarizationLM-Gemma-4-E4B-v1",    llm_gguf_filename="DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf",    llm_backend="gguf",    device="cpu",) transcript = pipe("conversation.wav", num_speakers=2)print(transcript.to_diarized_text())print(transcript.to_rttm(recording_id="conversation"))print(transcript.to_srt())

2. Python (`transformers` + `diarizationlm`)

First, install the required packages (Gemma4ForConditionalGeneration requires transformers>=5.0.0):

bash

pip install "transformers>=5.0.0" diarizationlm

Run inference on a GPU with bfloat16:

python

from diarizationlm import utilsimport torchfrom transformers import AutoModelForImageTextToText, AutoTokenizer MODEL_ID = "diarizers-community/DiarizationLM-Gemma-4-E4B-v1" HYPOTHESIS = (    "<speaker:1> Hello, how are you doing <speaker:2> today? I am doing well."    " What about <speaker:1> you? I'm doing well, too. Thank you.") print("Loading model...")tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, device_map="cuda")model = AutoModelForImageTextToText.from_pretrained(    MODEL_ID, torch_dtype=torch.bfloat16, device_map="cuda") print("Tokenizing input...")prompt = f"<|turn>user\n{HYPOTHESIS} --> <turn|>\n<|turn>model\n"inputs = tokenizer([prompt], return_tensors="pt").to("cuda") print("Generating completion...")outputs = model.generate(    **inputs,    max_new_tokens=int(inputs.input_ids.shape[1] * 1.2),    do_sample=False,    use_cache=True,) print("Decoding completion...")completion = tokenizer.batch_decode(    outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0]completion = utils.truncate_suffix_and_tailing_text(completion, " [eod]") print("Transferring completion to hypothesis text...")transferred_completion = utils.transfer_llm_completion(completion, HYPOTHESIS) print("========================================")print("Hypothesis:", HYPOTHESIS)print("========================================")print("Completion:", completion)print("========================================")print("Transferred completion:", transferred_completion)print("========================================")

3. GGUF (`llama.cpp`)

You can also run the quantized DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf model directly with llama.cpp:

bash

llama-cli \  -m DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf \  -p $'<|turn>user\n<speaker:1> Hello, how are you doing <speaker:2> today? I am doing well. What about <speaker:1> you? I\'m doing well, too. Thank you. --> <turn|>\n<|turn>model\n' \  --temp 0.0 \  -n 128

Citation

bibtex

@inproceedings{wang24h_interspeech,  title     = {{DiarizationLM: Speaker Diarization Post-Processing with Large Language Models}},  author    = {Quan Wang and Yiling Huang and Guanlong Zhao and Evan Clark and Wei Xia and Hank Liao},  year      = {2024},  booktitle = {Interspeech 2024},  pages     = {3754--3758},  doi       = {10.21437/Interspeech.2024-209},} @misc{akesbi-diarizers,  author = {Kamil Akesbi and Quan Wang and Sanchit Gandhi},  title = {"Diarizers: A repository for speaker diarization and DiarizationLM post-processing"},  year = {2024},  publisher = {GitHub},  journal = {GitHub repository},  howpublished = {\url{https://github.com/diarizers-community/diarizers}}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

diarizers-community

Model Tree

Base

google/gemma-4-E4B

Quantized

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Overview

DiarizationLM is a Large Language Model framework designed to post-process, correct, and optimize automatic speech recognition (ASR) and speaker diarization outputs.

Fisher English (2-speaker conversational telephone speech)
Callhome American English (2–5 speaker informal telephone conversations)
ICSI Meeting Corpus (3–9 speaker academic research meetings)
AMI Meeting Corpus (4-speaker tabletop meetings)

Foundation model: google/gemma-4-E4B
End-to-end open-source ASR + Diarization + DiarizationLM pipeline (diarizers): https://github.com/diarizers-community/diarizers (pip install diarizers)
Core DiarizationLM library & scripts (diarizationlm): https://github.com/google/speaker-id/tree/master/DiarizationLM (pip install diarizationlm)

Training Configuration

Base Architecture: Gemma 4 E4B (42 layers, hidden size 2560, hybrid 5:1 sliding-window and global attention, 262,144 vocabulary size)
LoRA Adapter: Rank r = 256 applied to all attention (q_proj, k_proj, v_proj, o_proj), MLP (gate_proj, up_proj, down_proj), and per-layer input (per_layer_input_gate, per_layer_projection, per_layer_model_projection) linear projections, merged into 16-bit (bfloat16) base weights and serialized to 4-bit GGUF (Q4_K_M)
Training Objective: Completion-only cross-entropy loss (<prompt> --> <completion> [eod])
Training Data: 51,063 Fisher + 20,762 Multi-Corpus (Callhome, ICSI, AMI) locality-preserving prompt-completion pairs
Optimization: 10,000 steps, global batch size 8, AdamW (beta1 = 0.9, beta2 = 0.99), peak learning rate 1.5e-4 with 500-step linear warmup and cosine decay on 8 Google Cloud TPU v5p chips
Prompt Segmentation Length: 4,000 characters (maximal sequence length 2,560 tokens)

Model Files Included

model.safetensors: Merged 16-bit (bfloat16) Hugging Face transformers weights (~16.0 GB)
DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf: (Recommended GGUF) Serialized 4-bit K-quant Medium (Q4_K_M) GGUF model (~5.30 GB) for llama.cpp / Ollama / llama-cpp-python (uses 256-element super-blocks in Q4_K with sensitive attn_v / ffn_down and embedding matrices retained in 6-bit Q6_K)
config.json, generation_config.json, tokenizer.json, tokenizer_config.json, processor_config.json, chat_template.jinja, special_tokens_map.json: Tokenizer and model configuration files

Benchmark Performance

1. USM + Turn-to-Diarize Frontend (with 95% Bootstrap Confidence Intervals)

Table with columns: Benchmark, Test Split, WER (%), System, WDER (%) [95% CI], cpWER (%) [95% CI], SpkCntMAE [95% CI], Paired ΔWDER vs. Baseline (p-value)
Benchmark	Test Split	WER (%)	System	WDER (%) [95% CI]	cpWER (%) [95% CI]	SpkCntMAE [95% CI]	Paired ΔWDER vs. Baseline (p-value)
Fisher	TEST FULL (172 sessions)	15.37	Baseline (USM + Turn-to-Diarize)`DiarizationLM-8b-Fisher-v2` (Llama 3 8B)`DiarizationLM-Gemma-4-E4B-v1` (4B)	5.32 [4.93, 5.74]3.282.99 [2.65, 3.37]	20.88 [19.97, 21.88]18.3717.62 [16.76, 18.56]	0.215 [0.151, 0.291]---0.093 [0.047, 0.145]	-------2.33% [-2.51, -2.16] (p < 0.0001)
Callhome	TEST FULL (20 calls)	15.22	Baseline (USM + Turn-to-Diarize)`DiarizationLM-8b-Fisher-v2` (Llama 3 8B)`DiarizationLM-Gemma-4-E4B-v1` (4B)	7.74 [6.07, 9.65]6.664.92 [3.46, 6.75]	24.31 [21.27, 27.43]23.5720.69 [17.98, 23.66]	0.050 [0.000, 0.150]---0.000 [0.000, 0.000]	-------2.82% [-3.48, -2.14] (p < 0.0001)
ICSI	TEST FULL (3 meetings)	29.42	Baseline (USM + Turn-to-Diarize)`DiarizationLM-Gemma-4-E4B-v1` (4B)	14.70 [11.65, 20.29]14.10 [10.77, 19.94]	43.90 [39.10, 51.67]43.32 [38.38, 51.20]	0.333 [0.000, 1.000]0.333 [0.000, 1.000]	----0.60% [-0.88, -0.35] (p < 0.0001)
AMI	TEST WORD FULL (16 meetings)	24.33	Baseline (USM + Turn-to-Diarize)`DiarizationLM-Gemma-4-E4B-v1` (4B)	15.68 [10.64, 21.11]14.89 [9.80, 20.38]	40.32 [32.43, 48.00]39.57 [31.57, 47.29]	0.500 [0.188, 0.812]0.438 [0.188, 0.750]	----0.79% [-1.00, -0.60] (p < 0.0001)

2. Open-Source Frontend (`diarizers` `v1.0.0` + `DiarizationLM-Gemma-4-E4B-v1`)

Table with columns: Benchmark (Full Test Split), Utterances, Pipeline / System, WER (%), WDER (%), cpWER (%), SpkCntMAE
Benchmark (Full Test Split)	Utterances	Pipeline / System	WER (%)	WDER (%)	cpWER (%)	SpkCntMAE
Fisher English	172	`diarizers` v1.0.0 Open Frontend (`Whisper-large-v3-turbo` + `pyannote/seg-3.0` + `ECAPA-TDNN` + `SpectralClusterer`, Fixed `K=2`)	20.80	4.35	24.78	0.000
Fisher English	172	`diarizers` v1.0.0 + `DiarizationLM-Gemma-4-E4B-v1` (`LG-TPST`)	20.80	2.59 (Raw-TPST: 2.65)	21.91 (Raw-TPST: 21.99)	0.000
CALLHOME English	20	`diarizers` v1.0.0 Open Frontend (Auto Speaker Count `[2..6]`, `eigen_threshold=0.45`)	21.25	9.42	32.42	0.050
CALLHOME English	20	`diarizers` v1.0.0 + `DiarizationLM-Gemma-4-E4B-v1`	21.25	4.89 (LG-TPST: 5.98)	25.91 (LG-TPST: 27.69)	0.050
ICSI Meeting Corpus	3	`diarizers` v1.0.0 Open Frontend (Auto Speaker Count `[2..10]`, `eigen_threshold=0.45`)	35.29	23.48	48.97	3.333
ICSI Meeting Corpus	3	`diarizers` v1.0.0 + `DiarizationLM-Gemma-4-E4B-v1` (`LG-TPST`)	35.29	23.10 (Raw-TPST: 23.11)	48.62 (Raw-TPST: 48.63)	3.333
AMI (`Mix-Headset`)	16	`diarizers` v1.0.0 Open Frontend (`AMI_MEETING_PRESET`, Auto Speaker Count `[3..5]`, `eigen_threshold=0.45`)	29.14	20.68	46.07	0.688
AMI (`Mix-Headset`)	16	`diarizers` v1.0.0 + `DiarizationLM-Gemma-4-E4B-v1`	29.14	20.13 (LG-TPST: 20.14)	45.53 (LG-TPST: 45.56)	0.688

Note: Post-LLM evaluation numbers above were computed with the full-precision (bfloat16) DiarizationLM-Gemma-4-E4B-v1 checkpoint using 6,000-character prompt chunks. For CPU deployment, this repository also provides DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf.

Usage

1. End-to-End Audio Pipeline (`diarizers`)

To transcribe, diarize, and post-process raw .wav / .mp3 audio end-to-end on CPU or GPU using the diarizers library:

bash

pip install "diarizers[audio,llm]"

python

import diarizers # CPU mode using the 4-bit Q4_K_M GGUF model (or pass llm_backend="transformers", device="cuda:0" on GPU):pipe = diarizers.DiarizersPipeline.from_pretrained(    asr_model="openai/whisper-large-v3-turbo",    diarizer_preset=diarizers.FISHER_TELEPHONY_PRESET,    llm_model_or_path="diarizers-community/DiarizationLM-Gemma-4-E4B-v1",    llm_gguf_filename="DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf",    llm_backend="gguf",    device="cpu",) transcript = pipe("conversation.wav", num_speakers=2)print(transcript.to_diarized_text())print(transcript.to_rttm(recording_id="conversation"))print(transcript.to_srt())

import diarizers # CPU mode using the 4-bit Q4_K_M GGUF model (or pass llm_backend="transformers", device="cuda:0" on GPU):pipe = diarizers.DiarizersPipeline.from_pretrained(    asr_model="openai/whisper-large-v3-turbo",    diarizer_preset=diarizers.FISHER_TELEPHONY_PRESET,    llm_model_or_path="diarizers-community/DiarizationLM-Gemma-4-E4B-v1",    llm_gguf_filename="DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf",    llm_backend="gguf",    device="cpu",) transcript = pipe("conversation.wav", num_speakers=2)print(transcript.to_diarized_text())print(transcript.to_rttm(recording_id="conversation"))print(transcript.to_srt())

2. Python (`transformers` + `diarizationlm`)

First, install the required packages (Gemma4ForConditionalGeneration requires transformers>=5.0.0):

bash

pip install "transformers>=5.0.0" diarizationlm

Run inference on a GPU with bfloat16:

python

from diarizationlm import utilsimport torchfrom transformers import AutoModelForImageTextToText, AutoTokenizer MODEL_ID = "diarizers-community/DiarizationLM-Gemma-4-E4B-v1" HYPOTHESIS = (    "<speaker:1> Hello, how are you doing <speaker:2> today? I am doing well."    " What about <speaker:1> you? I'm doing well, too. Thank you.") print("Loading model...")tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, device_map="cuda")model = AutoModelForImageTextToText.from_pretrained(    MODEL_ID, torch_dtype=torch.bfloat16, device_map="cuda") print("Tokenizing input...")prompt = f"<|turn>user\n{HYPOTHESIS} --> <turn|>\n<|turn>model\n"inputs = tokenizer([prompt], return_tensors="pt").to("cuda") print("Generating completion...")outputs = model.generate(    **inputs,    max_new_tokens=int(inputs.input_ids.shape[1] * 1.2),    do_sample=False,    use_cache=True,) print("Decoding completion...")completion = tokenizer.batch_decode(    outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0]completion = utils.truncate_suffix_and_tailing_text(completion, " [eod]") print("Transferring completion to hypothesis text...")transferred_completion = utils.transfer_llm_completion(completion, HYPOTHESIS) print("========================================")print("Hypothesis:", HYPOTHESIS)print("========================================")print("Completion:", completion)print("========================================")print("Transferred completion:", transferred_completion)print("========================================")

from diarizationlm import utilsimport torchfrom transformers import AutoModelForImageTextToText, AutoTokenizer MODEL_ID = "diarizers-community/DiarizationLM-Gemma-4-E4B-v1" HYPOTHESIS = (    "<speaker:1> Hello, how are you doing <speaker:2> today? I am doing well."    " What about <speaker:1> you? I'm doing well, too. Thank you.") print("Loading model...")tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, device_map="cuda")model = AutoModelForImageTextToText.from_pretrained(    MODEL_ID, torch_dtype=torch.bfloat16, device_map="cuda") print("Tokenizing input...")prompt = f"<|turn>user\n{HYPOTHESIS} --> <turn|>\n<|turn>model\n"inputs = tokenizer([prompt], return_tensors="pt").to("cuda") print("Generating completion...")outputs = model.generate(    **inputs,    max_new_tokens=int(inputs.input_ids.shape[1] * 1.2),    do_sample=False,    use_cache=True,) print("Decoding completion...")completion = tokenizer.batch_decode(    outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0]completion = utils.truncate_suffix_and_tailing_text(completion, " [eod]") print("Transferring completion to hypothesis text...")transferred_completion = utils.transfer_llm_completion(completion, HYPOTHESIS) print("========================================")print("Hypothesis:", HYPOTHESIS)print("========================================")print("Completion:", completion)print("========================================")print("Transferred completion:", transferred_completion)print("========================================")

3. GGUF (`llama.cpp`)

You can also run the quantized DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf model directly with llama.cpp:

bash

llama-cli \  -m DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf \  -p $'<|turn>user\n<speaker:1> Hello, how are you doing <speaker:2> today? I am doing well. What about <speaker:1> you? I\'m doing well, too. Thank you. --> <turn|>\n<|turn>model\n' \  --temp 0.0 \  -n 128

llama-cli \  -m DiarizationLM-Gemma-4-E4B-v1-q4_k_m.gguf \  -p $'<|turn>user\n<speaker:1> Hello, how are you doing <speaker:2> today? I am doing well. What about <speaker:1> you? I\'m doing well, too. Thank you. --> <turn|>\n<|turn>model\n' \  --temp 0.0 \  -n 128

Citation

bibtex

@inproceedings{wang24h_interspeech,  title     = {{DiarizationLM: Speaker Diarization Post-Processing with Large Language Models}},  author    = {Quan Wang and Yiling Huang and Guanlong Zhao and Evan Clark and Wei Xia and Hank Liao},  year      = {2024},  booktitle = {Interspeech 2024},  pages     = {3754--3758},  doi       = {10.21437/Interspeech.2024-209},} @misc{akesbi-diarizers,  author = {Kamil Akesbi and Quan Wang and Sanchit Gandhi},  title = {"Diarizers: A repository for speaker diarization and DiarizationLM post-processing"},  year = {2024},  publisher = {GitHub},  journal = {GitHub repository},  howpublished = {\url{https://github.com/diarizers-community/diarizers}}}

@inproceedings{wang24h_interspeech,  title     = {{DiarizationLM: Speaker Diarization Post-Processing with Large Language Models}},  author    = {Quan Wang and Yiling Huang and Guanlong Zhao and Evan Clark and Wei Xia and Hank Liao},  year      = {2024},  booktitle = {Interspeech 2024},  pages     = {3754--3758},  doi       = {10.21437/Interspeech.2024-209},} @misc{akesbi-diarizers,  author = {Kamil Akesbi and Quan Wang and Sanchit Gandhi},  title = {"Diarizers: A repository for speaker diarization and DiarizationLM post-processing"},  year = {2024},  publisher = {GitHub},  journal = {GitHub repository},  howpublished = {\url{https://github.com/diarizers-community/diarizers}}}

DiarizationLM-Gemma-4-E4B-v1

README

Overview

Training Configuration

Model Files Included

Benchmark Performance

1. USM + Turn-to-Diarize Frontend (with 95% Bootstrap Confidence Intervals)

2. Open-Source Frontend (diarizers v1.0.0 + DiarizationLM-Gemma-4-E4B-v1)

Usage

1. End-to-End Audio Pipeline (diarizers)

2. Python (transformers + diarizationlm)

3. GGUF (llama.cpp)

Citation

Explore FriendliAI today

README

Overview

Training Configuration

Model Files Included

Benchmark Performance

1. USM + Turn-to-Diarize Frontend (with 95% Bootstrap Confidence Intervals)

2. Open-Source Frontend (diarizers v1.0.0 + DiarizationLM-Gemma-4-E4B-v1)

Usage

1. End-to-End Audio Pipeline (diarizers)

2. Python (transformers + diarizationlm)

3. GGUF (llama.cpp)

Citation

2. Open-Source Frontend (`diarizers` `v1.0.0` + `DiarizationLM-Gemma-4-E4B-v1`)

1. End-to-End Audio Pipeline (`diarizers`)

2. Python (`transformers` + `diarizationlm`)

3. GGUF (`llama.cpp`)

2. Open-Source Frontend (`diarizers` `v1.0.0` + `DiarizationLM-Gemma-4-E4B-v1`)

1. End-to-End Audio Pipeline (`diarizers`)

2. Python (`transformers` + `diarizationlm`)

3. GGUF (`llama.cpp`)