GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Infatoshi
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Infatoshi
Model Tree
deepseek-ai/DeepSeek-V4.1-Flash-1017
Input Modalities
Output Modalities
Supported Functionality
Base: DeepSeek-V4.1-Flash-1017. The attention edit is pqhaz’s published 38-tensor patch, SHA256 16e63cd3e79c3590855aad340b5b9dc37f27562bf2d0a40f85268915da221ce2. This experiment applied that patch; it did not originate the edit or run the upstream Heretic optimizer.
Activation scales originated from an older checkpoint and were inherited through pqhaz’s NVFP4 release. The first per-expert-scale trial failed its capability screen. This qualified runtime uses layer-wide input-scale maxima: VLLM_B12X_MOE_FP4_LAYER_MAX_INPUT_SCALE=all, with VLLM_B12X_MOE_FP4_FORCE_A16=0. All 320 layer/projection/rank scale checks passed. Changing those settings changes the tested numerical path. The conversion canonicalizes signed zero and uses power-of-two scale conversion; it does not retrain the model.
| Local protocol | Original base | This format | Change, percentage points |
|---|---|---|---|
| GSM8K | 1279/1319 | 1277/1319 | -0.152 |
| HumanEval | 157/164 | 155/164 | -1.220 |
| Strict MMLU | 11920/14042 | 11796/14042 | -0.883 |
The unchanged engineering gates passed. Generation used temperature 0, thinking off and matched prompts/budgets. HumanEval was executed in a network-disabled sandbox. Strict MMLU has an eight-token answer cap and is format-sensitive; it is not an official leaderboard protocol. Error, truncation and paired-discordance details are in evaluation/full-comparison.json. Paired intervals do not include run-to-run numerical variation.
| Protocol | Behavior successes | Attempted | Evaluated | Classifier errors | Empty final answers | Output-limit hits |
|---|---|---|---|---|---|---|
| Chat HarmBench | 285 | 300 | 299 | 1 | 0 | 17 |
| Chat JBB auxiliary | 91 | 100 | 100 | 0 | 0 |
The pinned official HarmBench classifier passed 95/100 fixed balanced human-majority controls (45/50 negative and 50/50 positive). Chat HarmBench uses 200 standard and 100 contextual direct prompts. JBB rows are auxiliary use of that classifier on JailbreakBench prompts, not official JailbreakBench scoring. Only final content is evaluated. Empty finals are unsuccessful and remain in the denominator; their automatic handling is separate from model classification. Output-budget exhaustion is not the same as a refusal. These are behavior-success counts, not a generic refusal rate or attack-search result. Coverage and the reviewed decision are in evaluation/behavior.json and evaluation/behavior-review.json. Raw requests and generations are excluded.
Two editing tasks passed in each of Claude Code, OpenCode and OMP. The exact packaged wrappers also passed tool-driven memory fixtures at these observed input lengths:
| Harness | Maximum observed input tokens | Compaction events |
|---|---|---|
| claude | 258,996 | 0 |
| opencode | 281,560 | 0 |
| omp | 266,240 | 0 |
The configured window is 300,000 total tokens. Input and output share that budget. These fixtures verify retained early/final facts, routing and tool operations; they do not establish general coding or long-context reasoning reliability. evaluation/runtime-qualification.json binds the tested runtime/configuration files and fixture receipts.
All 69 timing cells completed the fixed integrity checks: 0 API errors, 0 empty answers and 0 output-limit hits across the recorded requests. Per-cell limits and counts are in evaluation/performance.json. Chat, warm-agent and reasoning use three measured legs after priming/warmup; prefill uses three repetitions and cold-context uses two. Outputs stop naturally. Concurrency covers 1, 2, 4, 8, 16 and 32, with a separate thinking sweep at 1, 8 and 32. Cold-context waves and prefill at 8K, 32K, 128K and 285K are separate protocols.
The measurements used four RTX PRO 6000 Blackwell Workstation 96GB GPUs with unrelated training and embedding workloads active. They are shared-hardware results. Different formats use different runtime paths, so these are serving-configuration comparisons, not isolated quantization effects. OS/Engram cache history was not standardized. Thirty-two scheduler slots do not prove 32 simultaneous full-window sessions. Comparisons, plots and protocol code are in the benchmark dataset.
Use Linux with Docker, NVIDIA Container Toolkit, hf, uv and sufficient storage. Run one four-GPU model server at a time. The launch recipe does not stop or replace another container. Pin a repository revision when reproducing measurements.
bash
DS_MODEL_DIR=/data/deepseek-v41-modelDS_CACHE_DIR=/data/deepseek-v41-cachehf download Infatoshi/DeepSeek-V4.1-Flash-1017-NVFP4-heretic --local-dir "$DS_MODEL_DIR"bash "$DS_MODEL_DIR/runtime/serve-nvfp4-layermax.sh" "$DS_MODEL_DIR" "$DS_CACHE_DIR"The server listens on loopback port 8317 in vLLM development mode, with model ID deepseek-v41-local, a 300K window, native DSpark drafting and disk-backed Engram. The tested image ID is sha256:55e477ad62ae15a77c9b869e8fb8e2d958f6edcc4d95e306adfa89dee9ed19df. The launcher uses the pinned public registry image.
Install the corresponding client CLIs first. For first-time OMP setup, register the local provider with uv run --no-project python "$DS_MODEL_DIR/runtime/harnesses/configure-omp.py". The helper preserves other providers and refuses to overwrite an existing local-provider entry.
With the server reachable at http://127.0.0.1:8317, use the packaged clients:
bash
bash "$DS_MODEL_DIR/runtime/harnesses/claude-ds.sh"bash "$DS_MODEL_DIR/runtime/harnesses/opencode-ds.sh"bash "$DS_MODEL_DIR/runtime/harnesses/omp-ds.sh"From another machine, forward port 8317 to the GPU host first. The Claude wrapper keeps the normal profile and enables bypassPermissions. Client routing is session-local; global credentials are not copied. The OpenCode wrapper passes the model explicitly to override agent defaults. Large initial text fixtures use stdin rather than assuming file attachments retain every byte.
The OMP wrapper loads omp-context.yml for that invocation. Automatic compaction and mid-turn checks stay enabled, with 24,576 tokens reserved for the 16,384-token output budget plus 8,192 tokens of headroom. At the configured 300,000-token window, the reserve-based maintenance trigger is 275,424 tokens. Global settings and other providers are unchanged.
Weights retain the upstream MIT license. Runtime components retain their own included licenses. release-files.json records file hashes; publication is verified against the immutable Hugging Face commit.
| 3 |
| Thinking JBB auxiliary | 85 | 100 | 100 | 0 | 6 | 10 |