GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
zeeshan141
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
zeeshan141
Model Tree
Qwen/Qwen3-VL-4B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Scored on the English records of SoMarkAI/doc-decision-bench (2,617 questions), using Docision's evaluator with --protocol none.
| Model | Overall | Document decisions | Industry classification |
|---|---|---|---|
| This model | 86.70% | 83.82% | 93.63% |
SoMarkAI/Docision-1.0 | 94.15% | 92.64% | 97.79% |
Qwen/Qwen3-VL-4B-Instruct | 83.57% | 78.68% | 95.32% |
This model doesn't beat Docision-1.0, and its classification score is below the base model's. The benchmark wasn't used for training, but it was used to choose between saved models, so these scores may be slightly optimistic. Chinese wasn't tested.
Generated multiple-choice questions built from DocVQA, ChartQA and RVL-CDIP. RVL-CDIP has known label noise of about 8%, and some ChartQA labels are wrong.
Needs a GPU and a version of transformers that supports Qwen3-VL. MODEL can also be a local folder.
python
import torchfrom PIL import Imagefrom transformers import AutoProcessor, Qwen3VLForConditionalGeneration MODEL = "zeeshan141/doc-decision" processor = AutoProcessor.from_pretrained(MODEL)# The same image size limits the benchmark scores were measured with.processor.image_processor.size = {"shortest_edge": 3136, "longest_edge": 501760}model = Qwen3VLForConditionalGeneration.from_pretrained(MODEL, dtype=torch.bfloat16, device_map="auto").eval() def choose(image_path, question, options, context="Inspect the supplied media carefully."): """Returns the chosen option and each option's probability.""" labels = [chr(ord("A") + i) for i in range(len(options))] listed = "".join(f"[{label}] {option}\n" for label, option in zip(labels, options)) text = f"{context}\nQuestion: {question}\nChoose exactly one candidate and answer with its label.\nCandidates:\n{listed}" with Image.open(image_path) as page: image = page.convert("RGB") messages = [{"role": "user", "content": [{"type": "image", "image": image}, {"type": "text", "text": text}]}] prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + "Answer:\n" inputs = processor(text=[prompt], images=[image], return_tensors="pt").to(model.device) with torch.no_grad(): logits = model(**inputs).logits[0, -1] label_ids = [processor.tokenizer.encode(label, add_special_tokens=False)[0] for label in labels] probabilities = logits[label_ids].float().softmax(-1).tolist() best = max(range(len(options)), key=probabilities.__getitem__) return options[best], dict(zip(options, probabilities)) answer, probabilities = choose("invoice.png", "Is the invoice total above $500?", ["Yes", "No"])print(answer, probabilities)The model reads the answer from the probabilities of the option labels, so it doesn't need generate(). Options must be different from each other, and the list can hold up to 26 with this code.
import torchfrom PIL import Imagefrom transformers import AutoProcessor, Qwen3VLForConditionalGeneration MODEL = "zeeshan141/doc-decision" processor = AutoProcessor.from_pretrained(MODEL)# The same image size limits the benchmark scores were measured with.processor.image_processor.size = {"shortest_edge": 3136, "longest_edge": 501760}model = Qwen3VLForConditionalGeneration.from_pretrained(MODEL, dtype=torch.bfloat16, device_map="auto").eval() def choose(image_path, question, options, context="Inspect the supplied media carefully."): """Returns the chosen option and each option's probability.""" labels = [chr(ord("A") + i) for i in range(len(options))] listed = "".join(f"[{label}] {option}\n" for label, option in zip(labels, options)) text = f"{context}\nQuestion: {question}\nChoose exactly one candidate and answer with its label.\nCandidates:\n{listed}" with Image.open(image_path) as page: image = page.convert("RGB") messages = [{"role": "user", "content": [{"type": "image", "image": image}, {"type": "text", "text": text}]}] prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + "Answer:\n" inputs = processor(text=[prompt], images=[image], return_tensors="pt").to(model.device) with torch.no_grad(): logits = model(**inputs).logits[0, -1] label_ids = [processor.tokenizer.encode(label, add_special_tokens=False)[0] for label in labels] probabilities = logits[label_ids].float().softmax(-1).tolist() best = max(range(len(options)), key=probabilities.__getitem__) return options[best], dict(zip(options, probabilities)) answer, probabilities = choose("invoice.png", "Is the invoice total above $500?", ["Yes", "No"])print(answer, probabilities)