Original Model Reference
Source: Index-Translate-Omni model card. The tables below reproduce the upstream results for the original models. They are not new benchmark results for this quantized export; see the format-specific verification below for the checks actually performed here.
Text Translation
Table with columns: Model, FLORES COMET22 ↑, WMT24++ COMET22 ↑, WMT26 Judge ↑, MEME ↑| Model | FLORES COMET22 ↑ | WMT24++ COMET22 ↑ | WMT26 Judge ↑ | MEME ↑ |
|---|
| Index-Translate 2B | 0.8655 | 0.8489 | 60.26 | 0.6443 |
| Index-Translate-Omni 2B | 0.8655 | 0.8489 | 60.26 | 0.6443 |
| Index-Translate 9B | 0.8789 | 0.8601 | 75.35 | 0.7387 |
| Index-Translate-Omni 9B | 0.8789 | 0.8601 | 75.35 | 0.7387 |
| Index-Translate 35BA3B (preview) | 0.8794 | 0.8586 | 76.76 | 0.7405 |
| Index-Translate-Omni 35BA3B | 0.8795 | 0.8593 | 78.01 | 0.7405 |
Instruction Following
Table with columns: Model, instTrans Quality / IFscore ↑, instTrans_minor Quality / IFscore ↑ / Off-target ↓, IFMTBench XCOMET-XXL / IFscore ↑| Model | instTrans Quality / IFscore ↑ | instTrans_minor Quality / IFscore ↑ / Off-target ↓ | IFMTBench XCOMET-XXL / IFscore ↑ |
|---|
| Index-Translate 2B | 0.5391 / 0.7569 | 0.3050 / 0.6586 / 3.97% | 0.7712 / 0.7584 |
| Index-Translate-Omni 2B | 0.5391 / 0.7569 | 0.3050 / 0.6586 / 3.97% | 0.7712 / 0.7584 |
| Index-Translate 9B | 0.6771 / 0.8209 | 0.5222 / 0.7725 / 3.47% | 0.7957 / 0.8760 |
| Index-Translate-Omni 9B | 0.6771 / 0.8209 | 0.5222 / 0.7725 / 3.47% | 0.7957 / 0.8760 |
| Index-Translate 35BA3B (preview) | 0.6901 / 0.8336 | 0.5151 / 0.7715 / 4.05% | 0.7926 / 0.8991 |
| Index-Translate-Omni 35BA3B | 0.6867 / 0.8371 | 0.5163 / 0.7843 / 3.62% | 0.7934 / 0.9070 |
Original Index-Translate figures come from the published benchmarks. The original 35BA3B reference is the published preview checkpoint.
Speech
CER is a percentage (lower is better). BLEU and chrF2 are higher-is-better. Translation cells show BLEU / chrF2 / target CER.
Table with columns: Model, Chinese ASR CER ↓, English ASR CER ↓, English → Chinese, Chinese → English| Model | Chinese ASR CER ↓ | English ASR CER ↓ | English → Chinese | Chinese → English |
|---|
| Index-Translate-Omni 2B | 13.8113 | 34.2560 | 36.982 / 30.357 / 43.3452 | 30.747 / 51.152 / 48.0670 |
| Index-Translate-Omni 9B | 17.8917 | 14.4097 | 43.507 / 36.312 / 41.3767 | 34.656 / 55.376 / 46.3720 |
| Index-Translate-Omni 35BA3B | 18.6803 | 15.0850 | 42.081 / 35.836 / 49.2490 | 36.404 / 57.323 / 45.1546 |
35BA3B uses a separate frontend for Chinese ASR; English ASR and speech translation use the native profile. Evaluation sample counts differ across model sizes.
Image Translation
COMET22 × 100 (higher is better). Each Omni model is compared with its Qwen3.5 baseline on paired evaluated subsets.
Table with columns: Model, MMTIT ↑, FLORES-rendered ↑, Vistra ↑| Model | MMTIT ↑ | FLORES-rendered ↑ | Vistra ↑ |
|---|
| Qwen3.5-2B | 52.164 | 50.187 | 55.326 |
| Index-Translate-Omni 2B | 65.809 | 53.612 | 65.596 |
| Qwen3.5-9B | 73.814 | 67.804 | 68.963 |
| Index-Translate-Omni 9B | 77.604 | 68.438 | 71.815 |
| Qwen3.5-35B-A3B | 75.602 | 70.005 | 71.067 |
| Index-Translate-Omni 35BA3B | 77.011 | 69.891 | 71.482 |
35BA3B reuses the verified image scores for the same weights. Results cover the evaluated subsets; the original failure and UNKNOWN scoring rules are retained.
FP4 Export
Quantized joint translation model with language and vision weights, original BF16 audio modules, and a portable runtime.
- Language: NVFP4 weights and calibrated activation scales, in 16-element blocks, exported with NVIDIA ModelOpt 0.44.
- Vision: original FP32 tensors, byte-identical to the assembled source.
- Audio: original BF16 audio encoder and connector, with a portable runtime.
lm_head, token embeddings, linear-attention modules and MoE routers are excluded from quantization. Ordinary MLP gate_proj layers are quantized.
Hardware and loading: Native NVFP4 inference requires a compatible Blackwell-generation backend. H100 is not Blackwell. The included JointModel uses fp4_reference.py and ModelOpt to unpack and dequantize the saved weights to BF16 for correctness testing; this needs BF16-sized memory and is not an accelerated FP4 serving path. Standard AutoModelForImageTextToText.from_pretrained should not be used directly on this ModelOpt export. Native Blackwell serving has not been validated here.
Loading
Download the entire repository, including audio/ and audio-tokenizer/:
pip install -r requirements-joint.txtpython - <<'PY'from joint_runtime import JointModel model = JointModel('.', device_map='auto')print(model.text_generate([{'role': 'user', 'content': 'Translate into English: 人工智能正在改变内容创作。'}]))print(model.audio_generate('<audio>\nTranscribe the speech and translate it into English.', ['/path/to/audio.wav']))PY
JointModel also supports image_generate(messages) with Transformers image-text messages. The runtime extracts only the required helpers from the supplied bound audio code; it does not require the original training repository. Keep the visual weights in FP32 and the audio weights in BF16.
Verification
Saved-checkpoint reload and text generation passed. The portable runtime also passed a text call, an image-description call, and one real WAV speech-to-English generation call on an H100. Text perplexity is 46.5939 over 250 held-out tokens using the reference loading mode above. These are small smoke tests, not a comprehensive translation, vision or speech-quality benchmark.
The tensor audit checked 96 quantized linear weights, 297 byte-identical visual tensors, 529 byte-identical audio tensors, and 225 byte-identical preserved language tensors. Exact outputs and limitations are in validation.json; file hashes are in SHA256SUMS.txt.
This repository does not include GGUF files; GGUF is a separate language/vision export without the end-to-end audio pipeline.
Mirrors