GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
caiovicentino1
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
caiovicentino1
Model Tree
caiovicentino1/Eikos-27B
Input Modalities
Output Modalities
Supported Functionality
Requires vLLM ≥ 0.30.0. Older builds return wrong answers when several long requests are batched together on this hybrid (Gated DeltaNet) architecture.
bash
hf download caiovicentino1/Eikos-27B-FP8 --local-dir Eikos-27B-FP8bash Eikos-27B-FP8/serve_vllm.sh $PWD/Eikos-27B-FP8 8001 # vLLM engine: letter readout + hybrid prefix cachepython Eikos-27B-FP8/serve.py --model $PWD/Eikos-27B-FP8 --vllm-url http://127.0.0.1:8001 --port 8000 # HTTP API on :8000The HTTP API, agent sessions and the question types are the same as for Eikos-27B.
Images (serve v1.3): this build reads images too; its vision tower is kept in bf16. Send them in
"images", as multipart files or inside the state: see Images for the
format, the limits and the full results. Zero-shot, on the same public items for both builds:
| 300 items per set | Eikos-27B (bf16) | Eikos-27B-FP8 |
|---|---|---|
| MME (yes/no) | 89.0% | 90.0% |
| MMStar | 70.7% | 70.7% |
| SEED-Bench-2-Plus | 74.0% | 74.3% |
| ScreenSpot-v2, no marks | 70.7% | 71.3% |
| ScreenSpot-v2, 4×4 grid drawn | 61.0% | 60.0% |
| ScreenSpot-Pro, no marks | 66.0% | 65.7% |
| Same answer as bf16: all / confident (≥0.9) | — | 97.6% / 100.0% |
Images are harder than our text suites: the bf16 model is at least 90% confident on only 25% of these items. Of the 44 answers (out of 1,800) that differ from bf16, all are on items where bf16 itself was below 0.7 confidence.
Same 7,371 items for both builds (7 suites, never used in training), vLLM 0.30 with batching and prefix cache on. The release gate was fixed before looking at results: accuracy within 1 point of bf16, ECE within 0.01, and at least 97% of answers unchanged.
| Eikos-27B (bf16) | Eikos-27B-FP8 | |
|---|---|---|
| Size | 55.6 GB | 31.2 GB |
| JevBench public — original / hard | 100.0 / 82.0 | 100.0 / 83.8 |
| DecisionBench — medium / hard | 89.1 / 78.2 | 88.7 / 78.8 |
| General battery (9 tasks) | 82.6 | 82.8 |
| Finance (CUAD, sentiment, FinQA-judge) | 85.4 | 85.3 |
| Trade rules — seen / unseen | 85.4 / 87.4 |
It passes our release gate.
We ran other groups' public benchmarks on this build, with their own harnesses and scorers, in September 2026. Official placement depends on each maintainer.
Decision Index 0.2.1: 38 benchmarks in five areas, each chance-corrected (0 = random guessing, 100 = perfect). We ran the full suite with the kit's own runner and scorer.
| Index | Knowledge & Reasoning | Language | Retrieval & Classification | Tools & Automation | Arts & Human Taste | |
|---|---|---|---|---|---|---|
| Jev | 57.89 | 51.3 | 62.0 | 55.4 | 75.1 | 37.7 |
| Eikos-27B-FP8 | 55.46 | 39.9 | 63.3 |
AgentRewardBench: judging whether a web agent completed its task. Test split, 1,106 trajectories, official scorer.
| Judge | Precision | Recall | F1 |
|---|---|---|---|
| Eikos-27B-FP8 | 78.9 | 73.6 | 76.1 |
| Jev (same inputs, through its API) | 75.7 | 65.4 | 70.2 |
| GPT-4o, Axtree (best published LLM judge) | 69.8 | 83.1 | 75.9 |
Jevals suite 0.1.0: Decision Score (100 = perfect, 0 = guessing the base rates). 300 items × 5 repeats per task, using the published states byte for byte.
| PubMedQA (yes/no) | Banking77 (choice, 77 options) | HelpSteer2 (score) | |
|---|---|---|---|
| Gemini 3.8 Flash | 73.0 | 74.1 | 4.6 |
| Eikos-27B-FP8 | 71.4 | 66.8 | 8.3 |
| Jev | 69.0 | 67.8 | 9.2 |
JevBench v1.4.2: a setup check with the stock typesafe adapter on the
231 public items gave easy 48/48, standard 72/72 and hard 93/111. The official run, which adds held-out and sealed
items, speed and cost, is requested: #117.
MIT for our contributions (LICENSE). The base model, Qwen3.8-27B, is Apache-2.0 (LICENSE-Qwen);
attributions are in NOTICE. Not legal, tax or investment advice.
| 85.6 / 86.9 |
| Compositional rules — same type / new domain / rulebooks | 95.6 / 94.3 / 95.3 | 95.6 / 94.7 / 95.0 |
| ECE (lower is better) | 0.043 | 0.042 |
| ≥0.90 confidence: decides / error | 44.1% / 2.5% | 44.2% / 2.6% |
| Same answer as bf16 (all / confident ≥0.9) | — | 98.8% / 100.0% |
| 74.4 |
| 39.8 |