GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ManniX-ITA
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ManniX-ITA
Model Tree
ManniX-ITA/gemma-4-A4B-98e-v7-coder-it
Input Modalities
Output Modalities
Supported Functionality
bash
python -m vllm.entrypoints.openai.api_server \--model ManniX-ITA/gemma-4-A4B-98e-v7-coder-NVFP4A16 \--served-model-name v7-coder \--port 8000 \--gpu-memory-utilization 0.92 \--max-model-len 65536 \--max-num-batched-tokens 8192 \--dtype bfloat16 \--trust-remote-code \--reasoning-parser gemma4 \--default-chat-template-kwargs '{"enable_thinking": true}'
Gemma 4 gotchas (all required): --max-num-batched-tokens 8192 (the MM-encoder
budget; the default 2048 < max_tokens_per_mm_item and crashes at boot),
--max-model-len 65536 (templates may request up to ~49k gen tokens), and the
gemma4 reasoning parser for the thinking format. NVFP4 needs a Blackwell/Hopper
or Ada GPU on a recent vLLM.
NVFP4A16 is a deployment format and is not separately benchmarked (cohort
policy). The table is the cohort's same-host Q6_K · llama.cpp · greedy
reference (temperature 0.0, top_p 1.0, top_k 0), read from summary.json;
it is representative of this model's quality. Row-max in bold. This repo =
v7-coder.
| Benchmark | 128e (unpruned) | v6-coder | v7-coder | v7-coderx |
|---|---|---|---|---|
| GPQA-diamond (198q) | 67.17 | 61.11 | 51.52 | 51.01 |
| AIME (30q) | 73.33 | 56.67 | 80.00 | 76.67 |
| MATH500 (100q) | 92.00 | 89.00 | 95.00 | 95.00 |
| GSM8K (100q) | 89.00 | 88.00 | 91.00 | 93.00 |
| ARC-Challenge (full) |
Metrics: GPQA & GSM8K = exact_match flexible-extract · MATH500 = math_verify ·
ARC & AIME = exact_match · IFEval = prompt_level_strict_acc · HumanEval/+ = pass@1
chat-extract · LCB-55/100 & MultiPL-E = pass@1. 128e uses the lcb_medium_55/100
templates; the prunes use lcb_medium_*_v4 (corrected harness, equivalent task). The
all-hard LCB-77 cross-model slice: 128e 79.22% / v7-coder 84.42% / v7-coderx 85.71%.
98e prune from 128e via the fkbroad code recipe (generate_drop_map_v5: generic_code
3×, targeted_lcb_medium_55 2×, all other targeting 0; target=98, protect_top=16,
alpha=2.0, strategy=max, no per-layer floor clamp), then the agentic loop-protection
force-keep (agentic_eog, 46 experts, 0/46 dropped — the loop fix that replaces fs2440), then
the mandatory shared-FFN α=1.2 upweight, then NVFP4A16 quantization. No targeted_gpqa term
and no DERN fold. Full recipe and the calibration-class table are on the
bf16 card.
A compact, vLLM-deployable Gemma 4 checkpoint for agentic coding and code
reasoning — the balanced code member of the v7-coder cohort (leads LCB-medium + HumanEval;
the code-maximal sibling v7-coderx
leads the all-hard LCB-77 and HE+). A research prune, not an official Google release;
generic_multilingual is de-weighted (0×) and graduate science (GPQA 51.52%) is a budget
axis. For llama.cpp/CPU deployment use the
GGUF repo.
128e → (v4 → v5 → v6-coder code line) → v7 competence-map rebuild → fkbroad code3/lcb2
selection + agentic loop-protection force-keep = v7-coder (loop-fixed; supersedes fs2440)
→ NVFP4A16. Built and evaluated on the omnimergekit toolchain.
| 95.39 |
| 92.15 |
| 86.60 |
| IFEval (100q, strict) | 97.00 | 92.00 | 92.00 | 92.00 |
| HumanEval (164) | 97.56 | 98.17 | 98.17 | 96.95 |
| HumanEval+ (164) | 92.07 | 92.68 | 92.07 | 93.29 |
| LCB-medium-55 v4 | 96.36 | 92.73 | 98.18 | 92.73 |
| LCB-medium-100 v4 | 97.00 | 94.00 | 94.00 | 91.00 |
| MultiPL-E (100) | 90.00 | 89.00 | 89.67 | 89.00 |