Quality
Test set. All non-canary segments of WMT24++, google/wmt24pp. That's 960 per direction and 8,640 in total,
covering news, social, speech and literary text. EN→X uses the original English source. PL→EN reverses the EN→PL
pair, so the Polish post-edited reference becomes the source and the original English becomes the reference.
Metrics. COMET-22 with Unbabel/wmt22-comet-da, scaled ×100, and corpus chrF from sacrebleu.
Setup. Greedy decoding on vLLM 0.26.0. One desktop RTX 5070 Ti 16 GB, capped at an 11.5 GB budget with
gpu_memory_utilization ≈ 0.74 to mimic a 12 GB laptop GPU.
Table with columns: EN→CS, EN→DE, EN→ES, EN→FR, EN→IT, EN→NL, EN→PL, EN→PT, PL→EN, All, chrF | EN→CS | EN→DE | EN→ES | EN→FR | EN→IT | EN→NL | EN→PL | EN→PT | PL→EN | All | chrF |
|---|
| BF16 reference | 83.01 | 82.15 | 83.61 | 81.63 | 84.07 | 83.84 | 83.26 | 82.66 | 81.14 | 82.82 | 58.7 |
| IndexTeam/Index-Translate-2B-FP8 | 82.94 | 82.15 | 83.66 | 81.57 | 83.99 | 83.83 | 83.23 | 82.61 | 81.15 | 82.79 | 58.6 |
| IndexTeam/Index-Translate-2B-FP4 | 81.47 | 81.73 | 83.28 | 81.07 | 83.88 | 83.13 | 82.35 | 82.47 | 80.80 | 82.24 | 57.9 |
| this model, NVFP4 GPTQ | 81.39 | 81.69 | 83.52 | 81.02 | 83.91 | 83.15 | 82.15 | 82.40 | 80.71 | 82.21 | 58.0 |
Czech loses the most, 1.6 points, in both 4-bit builds.
Structure test. 175 Excel-style cells in each of PL, DE, FR, ES and CS, with 270 tokens that must survive
verbatim: {placeholders}, %s, %1$s, {{var}}, ${var}, HTML tags, URLs, e-mails, SKUs, inline code and line
breaks. BF16, the official FP8 and this build all keep 97.0% of protected tokens and leave 95.4% of cells fully
intact. The official FP4 keeps 97.8%. The only misses are [PENDING]-style status labels, which the model
translates.
Throughput
4,000 short cells, EN→X and 1 to 40 words each, submitted in one batch with an 11.5 GB budget.
Table with columns: build, 128 sequences, 256 sequences, 512 sequences| build | 128 sequences | 256 sequences | 512 sequences |
|---|
| BF16 | 142 | not run | not run |
| IndexTeam/Index-Translate-2B-FP8 | 148 | 205 | 202 |
| IndexTeam/Index-Translate-2B-FP4 | 134 | not run | not run |
| this model | 144 | 243 | 268 |
The 128-sequence column ran on one RTX 5070 Ti and the rest on a second one, which measured about 5% faster on the
same workload.
On a 12 GB card, Hy-MT2-7B-NVFP4 scores 84.66 COMET-22 at 213
cells/s and is the better pick for the languages it covers. This model covers more languages and fits in less memory.
Usage with vLLM
Needs a Blackwell GPU for NVFP4. Tested with vLLM 0.26.0, text only.
vllm serve kurcontko/Index-Translate-2B-NVFP4 --language-model-only --max-model-len 4096 \ --gpu-memory-utilization 0.9 --max-num-seqs 512
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="-")prompt = "请将以下English文本翻译为Polish,直接输出翻译结果,不要进行任何解释。\n\n" + "Hello {name}, your order has shipped."r = client.chat.completions.create(model="kurcontko/Index-Translate-2B-NVFP4", temperature=0, messages=[{"role": "user", "content": prompt}], extra_body={"chat_template_kwargs": {"enable_thinking": False}})print(r.choices[0].message.content)
The prompt template comes from
bilibili/Index-Translate.
Always pass enable_thinking: false. If vLLM reports that max_num_seqs exceeds the available Mamba cache blocks,
lower --max-num-seqs.
If the engine fails at start-up while JIT-compiling FlashInfer's sampling kernels with
CUDA compiler and CUDA toolkit headers are incompatible
, set
VLLM_USE_FLASHINFER_SAMPLER=0.
Limitations
- Evaluated on the 9 directions above only, with COMET-22 and chrF. No human evaluation.
- Image translation and MTP speculative decoding are untested with this checkpoint.
License
Apache-2.0, same as IndexTeam/Index-Translate-2B.
Calibration data is OPUS-100.