febilly

Hy-MT2-1.8B-StreamRevise-v4-LoRA

README

License: apache-2.0

Prompt format

The adapter is trained on the layouts below. Use an explicit English source and target language name such as Japanese, Chinese, or English, and use greedy decoding.

First chunk of a new utterance

text
Translate the following text from {SOURCE_LANGUAGE} into {TARGET_LANGUAGE}. Note that you should only output the translated result without any additional explanation:

{CURRENT_SOURCE}

Subsequent updates

text
[Background Information]
Recent utterances and their translations:
{SOURCE_SENTENCE_1} → {TRANSLATION_1}
{SOURCE_SENTENCE_2} → {TRANSLATION_2}
...

Previous version of the current source:
{PREVIOUS_SOURCE}

Previous translation of the current source:
{PREVIOUS_TRANSLATION}

When the source meaning has not changed, preserve the still-correct prefix of the previous translation whenever possible. When content is added or corrected, accuracy and completeness take priority.

Please translate the following text from {SOURCE_LANGUAGE} into {TARGET_LANGUAGE}, taking the provided background information into consideration.

[Source Text]
{CURRENT_SOURCE}

The background blocks are optional, separated by one blank line, and kept in this order: recent context, previous source, previous translation, stability rule. Source-only history is also supported with this block:

text
Recent source utterances:
{SOURCE_SENTENCE_1}
{SOURCE_SENTENCE_2}
...

Use at most the most recent 10 utterances. Bilingual history is recommended when the application has the translations available. Do not mix bilingual and source-only lines inside the same history block.

Chat template

Use one user turn. apply_chat_template handles the wrapper; for a raw completion endpoint the exact form is:

text
<｜hy_begin▁of▁sentence｜><｜hy_User｜>{PROMPT}<｜hy_Assistant｜>

There is no trailing newline. Generation stops on token 120020.

Copy-paste renderer

python
LANGUAGE_NAMES = {
    "zh": "Chinese",
    "en": "English",
    "ja": "Japanese",
}

STABILITY_RULE = (
    "When the source meaning has not changed, preserve the still-correct prefix "
    "of the previous translation whenever possible. When content is added or "
    "corrected, accuracy and completeness take priority."
)


def render_prompt(*, source_language, target_language, current_source,
                  history=(), previous_source="", previous_translation="",
                  stability_rule=True):
    source = LANGUAGE_NAMES.get(source_language, source_language)
    target = LANGUAGE_NAMES.get(target_language, target_language)

    if not history and not previous_source and not previous_translation:
        return (
            f"Translate the following text from {source} into {target}. Note that you "
            "should only output the translated result without any additional "
            f"explanation:\n\n{current_source}"
        )

    blocks = []
    if history:
        bilingual = isinstance(history[0], (tuple, list))
        if bilingual:
            lines = [f"{src} → {tgt}" for src, tgt in history[-10:]]
            blocks.append("Recent utterances and their translations:\n" + "\n".join(lines))
        else:
            blocks.append("Recent source utterances:\n" + "\n".join(history[-10:]))
    if previous_source:
        blocks.append("Previous version of the current source:\n" + previous_source)
    if previous_translation:
        blocks.append("Previous translation of the current source:\n" + previous_translation)
    if stability_rule:
        blocks.append(STABILITY_RULE)

    return (
        "[Background Information]\n" + "\n\n".join(blocks)
        + f"\n\nPlease translate the following text from {source} into {target}, "
        "taking the provided background information into consideration."
        + "\n\n[Source Text]\n" + current_source
    )

Streaming example

An actual greedy Q4_K_M run on an English-to-Chinese ASR trajectory:

Table with columns: step, current_source, previous_translation, output
step	`current_source`	`previous_translation`	output
1	`It's too small,`	(none)	`它太小了，`
2	`It's too small, won't go into supernova,`	`它太小了，`	`它太小了，不会变成超新星，`

Very short fragments can intentionally produce an empty string when there is not enough information to translate safely. In that case, keep displaying the last non-empty translation or wait for the next ASR update.

Usage

python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "tencent/Hy-MT2-1.8B"
ADAPTER = "febilly/Hy-MT2-1.8B-StreamRevise-v4-LoRA"

tokenizer = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    BASE, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, ADAPTER).eval()


def translate(prompt: str) -> str:
    text = tokenizer.apply_chat_template(
        [{"role": "user", "content": prompt}],
        tokenize=False,
        add_generation_prompt=True,
    )
    inputs = tokenizer(text, return_tensors="pt").to(model.device)
    with torch.inference_mode():
        output = model.generate(
            **inputs,
            do_sample=False,
            max_new_tokens=256,
            eos_token_id=120020,
            pad_token_id=tokenizer.pad_token_id,
        )
    return tokenizer.decode(
        output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True
    ).strip()


prompt = render_prompt(
    source_language="en",
    target_language="zh",
    current_source="It's too small, won't go into supernova.",
    history=[("That star is faint.", "那颗恒星很暗。")],
    previous_source="It's too small,",
    previous_translation="它太小了，",
)
print(translate(prompt))

Decoding: use greedy

Set do_sample=False (equivalently temperature=0). Consecutive ASR updates produce nearly identical prompts, so sampling creates unrelated output changes that appear as subtitle flicker. The evaluation below uses greedy decoding.

Driving it as a stream

The model is stateless. The caller owns the current revision chain:

python
history = []
previous_source = ""
previous_translation = ""

for current_source in asr_updates():
    output = translate(render_prompt(
        source_language="ja",
        target_language="zh",
        current_source=current_source,
        history=history,
        previous_source=previous_source,
        previous_translation=previous_translation,
    ))

    if output:
        display(output)
        previous_translation = output
    previous_source = current_source

    if utterance_finished:
        if previous_translation:
            history.append((current_source, previous_translation))
            history = history[-10:]
        previous_source = ""
        previous_translation = ""

Enable prefix KV caching when your inference backend supports it. Requests remain independently retryable and load-balanceable because no state is stored by the model.

Evaluation

The published Q4_K_M build was evaluated with greedy decoding. The gate combines three independent tracks: general translation quality, real streaming ASR behavior, and targeted regression probes.

FLORES-200 devtest

Six Chinese/English/Japanese directions, 1,012 examples per direction. The metric is corpus chrF with word_order=0; the overall corpus score is 46.393.

Table with columns: direction, chrF
direction	chrF
Japanese → Chinese	27.303
Chinese → Japanese	32.953
English → Chinese	38.202
Chinese → English	57.369
Japanese → English	54.671
English → Japanese	40.643

FLORES is English-pivoted, so absolute scores for Chinese↔Japanese include reference divergence. Use these numbers as reproducible test results, not as a direct ranking of language-pair difficulty.

Real streaming replay

500 utterance trajectories / 792 ASR states:

Table with columns: metric, result
metric	result
source-language leakage	1.01%
true source-copy rate	0.00%
empty-output rate for source fragments longer than 4 characters	0.35%
punctuation-insensitive prefix retention	0.950
punctuation-insensitive characters erased per transition	0.272
punctuation-insensitive rewrite rate	6.8%
final output/source length ratio, median	0.795

The targeted no-copy, no-added-brackets, and word-sense probes passed 24/24 cases. Streaming metrics are application-oriented heuristics and are not directly comparable with simultaneous-MT paper benchmarks.

Training

Table with columns: item, value
item	value
Base	`tencent/Hy-MT2-1.8B` (HunYuanDenseV1, 32 layers, hidden size 2048)
Method	LoRA, `r=64`, `alpha=128`, `dropout=0.05`
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`

Data construction

The training set contains 36,611 streaming-revision states and 12,167 standalone translation anchors, split nearly evenly across all six directions between Chinese, English, and Japanese.

Streaming examples are built as trajectories of realistic ASR states. About 80% of transitions are prefix extensions, with the rest covering punctuation changes and substantive re-recognition. The data includes punctuation-triggered updates, pause-triggered multi-character updates, and very short fragments. Recent context is represented by bilingual source/translation pairs in most contextual samples, with source-only history retained as a supported input form.

Target construction uses two stages: a final translation is produced independently for the complete source, then intermediate translations are generated for partial ASR states. A separate future-content leakage check regenerates suspect intermediate states without access to the final translation. Standalone anchors are generated independently without streaming background so ordinary one-shot translation remains supported.

The training dataset is not released.

Limitations

Stability is a tendency, not a guarantee. A recognition correction can require rewriting the whole line.
Very short fragments can return an empty string. Keep the prior non-empty subtitle until more source text arrives.
Prompt format matters. Off-format prompts can reduce both translation quality and stability.
Core language coverage is Chinese, English, and Japanese. Other directions were not trained or evaluated.
Sentence-scoped revision. The model revises only the current utterance; finalized earlier subtitles are not revisited.
Greedy decoding is assumed.
Garbled or highly incomplete ASR text can still produce mistranslations or hallucinations.
The adapter inherits the base model's biases and limitations.

License

Apache 2.0, the same license as tencent/Hy-MT2-1.8B.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider