GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
DanVP
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
DanVP
Model Tree
Qwen/Qwen3.5-0.8B
Input Modalities
Output Modalities
Supported Functionality
| File | Size | Use |
|---|---|---|
model.safetensors (+ model-extra.safetensors: the MTP head of the base) | 1.75 GB | transformers / vLLM, bf16. The vision tower is the base model's, unchanged and not evaluated |
Qwen3.5-0.8B-zh-en-novel-BF16.gguf | 1.52 GB | llama.cpp, text only |
Qwen3.5-0.8B-zh-en-novel-Q8_0.gguf | 0.81 GB | llama.cpp |
Qwen3.5-0.8B-zh-en-novel-Q6_K.gguf | 0.63 GB | llama.cpp |
Qwen3.5-0.8B-zh-en-novel-Q4_K_M.gguf | 0.53 GB | llama.cpp |
banned_token_ids.json | 103,492 token ids to ban at decode time | |
prompts.py | the exact prompt formats of training (build_prompt, render, split_windows) |
GGUF files: llama.cpp b12015, convert_hf_to_gguf.py --outtype bf16 --no-mtp, then llama-quantize without an importance matrix.
Judge: Claude Haiku 5.5, blind to the system, all systems in one shuffled batch; each output judged 3 times, label = majority. Paired difference, bootstrap 10,000, 95 % CI. Sets: P-600, 600 held-out lines from the training pool (judge: is the translation faithful? "bad" = omission, addition or mistranslation); G2-J300, 300 lines of the GuoFeng web-novel test set judged against the human reference (correct / minor / wrong); G2 chapters, 179 windows of 3–5 lines; glossary probe, 200 lines with a gender note for one character, each in both genders.
| This model (bf16, vLLM, ban) | Index-Translate-2B | Difference | |
|---|---|---|---|
| P-600 bad | 9.67 % | 6.00 % | +3.67 [+1.00; +6.33] |
| G2-J300 wrong | 16.33 % | 11.33 % | +5.00 [+0.67; +9.33] |
| G2-J300 correct | 48.33 % | 56.33 % | −8.00 [−13.67; −2.33] |
| Chapter windows with the right line count | 100 % | 97.2 % | |
| Lines with Han characters (1,816 lines) | 0 % | 0.72 % | |
| Glossary probe: follows the requested gender (he / she) | 90.0 % / 87.0 % | 85.0 % / 84.0 % |
Errors that remain are mostly in reading the Chinese: idioms and slang, rhetorical questions read as real questions, who does what to whom, rare characters, long sentences.
| File | P-600 bad | G2-J300 wrong | Window line count | CPU tok/s (1 stream / 8 slots) |
|---|---|---|---|---|
| BF16 | 11.33 % | 16.00 % | 100 % | 38.8 / 86 |
| Q8_0 | −0.33 [−1.83; +1.17] | +1.33 [+0.33; +2.67] | 100 % | 61.5 / 105 |
| Q6_K | +0.33 [−1.33; +2.17] | −0.67 [−3.00; +1.67] | 100 % | 68.3 / 115 |
| Q4_K_M | +1.17 [−0.83; +3.00] | +4.33 [+0.67; +7.67] | 100 % | 74.9 / 122 |
CPU: Intel Core Ultra 9 275HX, 16 threads. Q6_K shows no measurable loss; Q4_K_M loses about 4 points on the G2 wrong rate. The BF16 GGUF and the bf16 vLLM model give the same judged quality (P-600 bad +0.17, CI [−1.0; +1.3]).
No system prompt, and the empty think block of the Qwen3.5 template (enable_thinking=False). prompts.render() builds the same string.
text
Single line:请将以下中文文本翻译为英语，直接输出翻译结果，不要进行任何解释。\n\n{line} Window of 2–5 lines (keeps the line count), optional glossary and gender note:请将以下中文小说翻译成英语，并且严格遵循所有约束要求。 【源文】{line 1}{line 2} 【约束要求】1. 【硬性要求】保持与源文相同的行数，逐行对应，不要合并或拆分行2. 【注意】这是小说的叙述文字，请使用过去时叙述；人物对话和引文内的时态保持自然3. 【硬性要求】专名/术语对照: 林君→Lin Jun、呵呵→Hehe4. 【注意】人物性别：林君为男性，用 he/his 只输出译文，不要有任何额外说明。For a chapter, split the text into windows of at most 5 lines (prompts.split_windows). Max new tokens: 512 for a line; 3 × source
tokens + 64 for a window. Stop on <|im_end|> (248046) or <|endoftext|> (248044). Greedy decoding.
bash
llama-server -m Qwen3.5-0.8B-zh-en-novel-Q6_K.gguf -c 4096python
# pip install huggingface_hub requests; put prompts.py and banned_token_ids.json next to this script:# huggingface-cli download DanVP/Qwen3.5-0.8B-zh-en-novel prompts.py banned_token_ids.json --local-dir .import json, requestsfrom prompts import build_prompt, renderban = [[i, False] for i in json.load(open("banned_token_ids.json"))]body = {"prompt": render(build_prompt(["他来了。"])), "n_predict": 512, "temperature": 0, "top_k": 1, "stop": ["<|im_end|>", "<|endoftext|>"], "logit_bias": ban, "cache_prompt": False}print(requests.post("http://127.0.0.1:8080/completion", json=body).json()["content"])Our evaluation sent the prompt as token ids from the Hugging Face tokenizer (tokenizer(render(p), add_special_tokens=False)).
python
# prompts.py and banned_token_ids.json from this repo next to the script (see the llama.cpp example)import json, torchfrom transformers import AutoTokenizer, Qwen3_5ForConditionalGenerationfrom prompts import build_prompt, renderrepo = "DanVP/Qwen3.5-0.8B-zh-en-novel"tok = AutoTokenizer.from_pretrained(repo)model = Qwen3_5ForConditionalGeneration.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")ban = json.load(open("banned_token_ids.json"))lines = ["林君脸上不喜不忧，一脸平淡。", "“白校长，看我又带一个回来了！”"]ids = tok(render(build_prompt(lines, window=True)), add_special_tokens=False, return_tensors="pt").to(model.device)out = model.generate(**ids, max_new_tokens=256, do_sample=False, suppress_tokens=ban)print(tok.decode(out[0, ids["input_ids"].shape[1]:], skip_special_tokens=True))SamplingParams(temperature=0, allowed_token_ids=[ids not in banned_token_ids.json]). vLLM 0.29 caps allowed_token_ids at 1,024 ids in
its default (V2) model runner; set VLLM_USE_V2_MODEL_RUNNER=0 to use the V1 runner, which masks the full vocabulary.
<|im_end|>. Dev cross-entropy 0.504 at the end.Apache-2.0. Base model: Qwen3.5-0.8B (Qwen team, Apache-2.0). Training targets were produced with Index-Translate-9B and Index-Translate-35B-A3B-preview (bilibili Index team, Apache-2.0).
Model 0.8B dịch truyện mạng Trung → Anh, SFT từ Qwen3.5-0.8B trên bản dịch của Index-Translate-9B / 35B. Kém Index-Translate-2B khoảng 3–4
điểm dòng sai (đánh giá bằng judge LLM), hơn ở giữ số dòng và theo ghi chú giới tính. Luôn decode với danh sách token cấm
banned_token_ids.json để loại chữ Hán sót. Bản GGUF: Q6_K không thấy giảm chất lượng; Q4_K_M giảm khoảng 4 điểm ở tỉ lệ dòng sai của G2.
Dùng thử trong trình duyệt: Space DanVP/Qwen3.5-0.8B-zh-en-novel-demo
chạy bản Q6_K trên GPU của máy bạn (WebGPU), lần đầu tải khoảng 636 MB.
Single line:请将以下中文文本翻译为英语，直接输出翻译结果，不要进行任何解释。\n\n{line}Window of 2–5 lines (keeps the line count), optional glossary and gender note:请将以下中文小说翻译成英语，并且严格遵循所有约束要求。【源文】{line 1}{line 2}【约束要求】1. 【硬性要求】保持与源文相同的行数，逐行对应，不要合并或拆分行2. 【注意】这是小说的叙述文字，请使用过去时叙述；人物对话和引文内的时态保持自然3. 【硬性要求】专名/术语对照: 林君→Lin Jun、呵呵→Hehe4. 【注意】人物性别：林君为男性，用 he/his只输出译文，不要有任何额外说明。# pip install huggingface_hub requests; put prompts.py and banned_token_ids.json next to this script:# huggingface-cli download DanVP/Qwen3.5-0.8B-zh-en-novel prompts.py banned_token_ids.json --local-dir .import json, requestsfrom prompts import build_prompt, renderban = [[i, False] for i in json.load(open("banned_token_ids.json"))]body = {"prompt": render(build_prompt(["他来了。"])), "n_predict": 512, "temperature": 0, "top_k": 1, "stop": ["<|im_end|>", "<|endoftext|>"], "logit_bias": ban, "cache_prompt": False}print(requests.post("http://127.0.0.1:8080/completion", json=body).json()["content"])# prompts.py and banned_token_ids.json from this repo next to the script (see the llama.cpp example)import json, torchfrom transformers import AutoTokenizer, Qwen3_5ForConditionalGenerationfrom prompts import build_prompt, renderrepo = "DanVP/Qwen3.5-0.8B-zh-en-novel"tok = AutoTokenizer.from_pretrained(repo)model = Qwen3_5ForConditionalGeneration.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")ban = json.load(open("banned_token_ids.json"))lines = ["林君脸上不喜不忧，一脸平淡。", "“白校长，看我又带一个回来了！”"]ids = tok(render(build_prompt(lines, window=True)), add_special_tokens=False, return_tensors="pt").to(model.device)out = model.generate(**ids, max_new_tokens=256, do_sample=False, suppress_tokens=ban)print(tok.decode(out[0, ids["input_ids"].shape[1]:], skip_special_tokens=True))