GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ncoder-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ncoder-ai
Model Tree
rsxdalv/VibeVoice-Large
Input Modalities
Output Modalities
Supported Functionality
For advanced users hand-grafting the AWQ Qwen2 into a custom base. You provide the FP16 audio stack (acoustic tokenizer, diffusion head, connectors); this repo provides only the quantized language model.
python
import torch, gcfrom vibevoice.modular.modeling_vibevoice_inference import VibeVoiceForConditionalGenerationInferencefrom awq import AutoAWQForCausalLM # Your custom FP16 basemodel = VibeVoiceForConditionalGenerationInference.from_pretrained( "rsxdalv/VibeVoice-Large", torch_dtype=torch.float16, device_map="cuda:0",).eval() # Free FP16 LLM, graft AWQ Qwen2 in its placedel model.model.language_modelgc.collect(); torch.cuda.empty_cache() awq = AutoAWQForCausalLM.from_quantized( "ncoder-ai/VibeVoice-Large-AWQ-INT4", device_map={"": 0}, safetensors=True, fuse_layers=False,)model.model.language_model = awq.model.modeldel awq; gc.collect(); torch.cuda.empty_cache()Quantization recipe: AutoAWQ, 4-bit, group_size=128, GEMM (Marlin) version, zero_point=True. Calibration: 250 samples (200 prose + 50 wikitext), 512-token max length.
MIT — same as upstream rsxdalv/VibeVoice-Large.