Quality
Test set. All non-canary segments of WMT24++, google/wmt24pp. That's 960 per direction and 8,640 in total,
covering news, social, speech and literary text. EN→X uses the original English source. PL→EN reverses the EN→PL
pair, so the Polish post-edited reference becomes the source and the original English becomes the reference.
Metrics. COMET-22 with Unbabel/wmt22-comet-da, scaled ×100, and corpus chrF from sacrebleu.
Setup. Greedy decoding on vLLM 0.26.0. One desktop RTX 5070 Ti 16 GB, capped at an 11.5 GB budget with
gpu_memory_utilization ≈ 0.74 to mimic a 12 GB laptop GPU.
Table with columns: EN→CS, EN→DE, EN→ES, EN→FR, EN→IT, EN→NL, EN→PL, EN→PT, PL→EN, All, chrF | EN→CS | EN→DE | EN→ES | EN→FR | EN→IT | EN→NL | EN→PL | EN→PT | PL→EN | All | chrF |
|---|
| IndexTeam/Index-Translate-9B-FP8 on 16 GB | 86.78 | 83.88 | 84.70 | 83.22 | 85.83 | 85.65 | 86.08 | 83.52 | 82.49 | 84.68 | 60.4 |
| this model, NVFP4 at 11.5 GB | 86.58 | 83.80 | 84.79 | 83.14 | 85.59 | 85.65 | 85.84 | 83.47 | 82.48 | 84.59 | 60.1 |
Structure test. 175 Excel-style cells in each of PL, DE, FR, ES and CS, with 270 tokens that must survive
verbatim: {placeholders}, %s, %1$s, {{var}}, ${var}, HTML tags, URLs, e-mails, SKUs, inline code and line
breaks. With the official prompt this model keeps 97.0% of protected tokens and leaves 95.4% of cells fully intact.
Throughput
Each Gated DeltaNet layer holds a fixed-size state for every running sequence. Under an 11.5 GB budget vLLM fits
about 48 concurrent sequences, and that caps batched throughput. In a 4,000-cell test with --max-num-seqs 48 this
model translated 39.3 short cells/s.
Hy-MT2-7B-NVFP4 did 213.4 cells/s on the same test, about 5×
faster, so use it for high-volume batch work on 12 GB. Pick this model when you need a language Hy-MT2 doesn't
cover, or when you can't change the prompt and still need placeholders and line breaks kept.
Usage with vLLM
Needs a Blackwell GPU for NVFP4. Tested with vLLM 0.26.0, text only.
vllm serve kurcontko/Index-Translate-9B-NVFP4 --language-model-only --max-model-len 4096 \ --gpu-memory-utilization 0.9 --max-num-seqs 32
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="-")prompt = "请将以下English文本翻译为Polish,直接输出翻译结果,不要进行任何解释。\n\n" + "Hello {name}, your order has shipped."r = client.chat.completions.create(model="kurcontko/Index-Translate-9B-NVFP4", temperature=0, messages=[{"role": "user", "content": prompt}], extra_body={"chat_template_kwargs": {"enable_thinking": False}})print(r.choices[0].message.content)
The prompt template comes from
bilibili/Index-Translate.
Always pass enable_thinking: false. If vLLM reports that max_num_seqs exceeds the available Mamba cache blocks,
lower --max-num-seqs.
If the engine fails at start-up while JIT-compiling FlashInfer's sampling kernels with
CUDA compiler and CUDA toolkit headers are incompatible
, set
VLLM_USE_FLASHINFER_SAMPLER=0.
Limitations
- Evaluated on the 9 directions above only, with COMET-22 and chrF. No human evaluation.
- Image translation and MTP speculative decoding are untested with this checkpoint.
License
Apache-2.0, same as IndexTeam/Index-Translate-9B.
Calibration data is OPUS-100.