GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
leinenterprises
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
leinenterprises
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
Decision Index 0.3, public vision suite (12,210 rows, all answered), official kit:
| Benchmark | jev-vision-27b | Herbert v0.5 | Herbert v0.8 |
|---|---|---|---|
| CV-Bench | 76.8 | 79.2 | 81.5 |
| BLINK | 54.6 | 72.8 | 76.2 |
| RealWorldQA | 72.8 | 73.4 | 72.5 |
| CharXiv | 78.9 | 77.4 | 77.9 |
| InfographicVQA | 96.2 | 96.7 | 96.2 |
| Mind2Web | 84.4 | 83.1 | 86.3 |
| Winoground | 83.5 | 84.0 | 83.0 |
| KIE (CORD + FUNSD) | 91.3 | 92.5 | 92.5 |
| Hateful Memes | 52.0 | 52.7 | 50.4 |
| R-Bench-M | 24.7 | 31.2 | 35.4 |
| MMMU-Pro vision | 38.0 | 42.3 | 42.5 |
| Public vision index | 73.06 | 75.69 | 76.38 |
Run files: leinenterprises/herbert-results.
jev-bench (22,773 independent public decisions, scored with Jevify, base in 4-bit): macro accuracy 0.747, ECE 0.074 (jev-vision-27b 0.734 / 0.084; Jev 1.13.0 0.733 / 0.113 per the Jevify README). Run files.
Text: on a fixed 5,018-question sample of the public text suite (42 benchmarks), v0.8 scores within 0.5 points on average of its text-trained intermediate stage; the full text suite has not been run on this version yet.
Options are labelled A, B, C, ... in the prompt; the answer is read from the label logits at the answer position
and softmaxed over the listed options only. Up to 624 options per question (A-Z, a-z, 0-9, then single-token
two-letter labels); larger questions are refused as unsupported, never truncated. Images up to 8 per request.
bash
pip install vllm fastapi uvicorn peft transformers "decision-index[vision] @ git+https://github.com/apolinario/decision-index"cd codepython -m train.merge --ckpt .. --out ../merged # adapter -> plain bf16 model (one-time)python -m serve.server --model ../merged --perms 1 --port 8000 --max-model-len 65536 --max-pixels 1638400POST /v1/systemone (Decision Index wire format) or /v1/decisions (the kit's vision decide engine, images as a
separate list).
LoRA r=32, alpha=64 on the language model; vision tower frozen. Starting from Herbert v0.5:
No benchmark items or evaluation splits were used, and every training image was near-duplicate filtered (perceptual hash) against the evaluation images.
Public-suite numbers are self-run with the official kit; the board's Full score also includes private tests run by the maintainers. Text is measured on a sample only; on the public text board this family trails text-first models.
pip install vllm fastapi uvicorn peft transformers "decision-index[vision] @ git+https://github.com/apolinario/decision-index"cd codepython -m train.merge --ckpt .. --out ../merged # adapter -> plain bf16 model (one-time)python -m serve.server --model ../merged --perms 1 --port 8000 --max-model-len 65536 --max-pixels 1638400