GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
p4b
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
p4b
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Input:
markdown
<src>First sentence.[|1|]Second sentence.[|2|]Third.[|3|]</src><tgt>첫 문장.[|1|]두 번째 문장.[|2|]세 번째.[|3|]</tgt>
Output — cumulative SourceIndex-TargetIndex boundary pairs:
markdown
<answer>1-1, 2-2, 3-3</answer>
This is a text model. It is packaged in the Qwen3_5ForConditionalGeneration
layout (the original vision tower is bundled unused) so it loads in both Hugging Face
Transformers and vLLM; no images are involved.
bash
vllm serve p4b/qwen3.5-4b-chunky-NVFP4A16 --language-model-only --mamba-cache-mode=align
--language-model-only skips the (unused) vision tower; --mamba-cache-mode=align
is required by the Qwen3.5 hybrid linear-attention layers.
python
from transformers import AutoModelForImageTextToText, AutoTokenizertok = AutoTokenizer.from_pretrained("p4b/qwen3.5-4b-chunky-NVFP4A16")model = AutoModelForImageTextToText.from_pretrained("p4b/qwen3.5-4b-chunky-NVFP4A16", device_map="auto")msgs = [{"role": "user", "content": PROMPT + text}] # text = the <src>...</tgt> blockenc = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)out = model.generate(input_ids=enc["input_ids"].to(model.device), max_new_tokens=128)print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))
Apache-2.0.