GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
littlemex
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
littlemex
Model Tree
Qwen/Qwen3.6-35B-A3B-FP8
Input Modalities
Output Modalities
Supported Functionality
bash
pip install "prismyra[server,fast] @ git+https://github.com/littlemex/Prismyra@main"bash
PRISMYRA_EXPERTS=nvfp4 \PRISMYRA_NVFP4_EXPERTS=/path/to/nvfp4_experts.safetensors \PRISMYRA_NVFP4_CALIB=/path/to/nvfp4_calib.json \PRISMYRA_DENSE=nvfp4 \PRISMYRA_DENSE_KEEP='self_attn|shared_expert' \prismyra-serve --model /path/to/this/checkpoint/directoryThe checkpoint directory and the two side files (nvfp4_experts.safetensors, nvfp4_calib.json) are downloaded from
this repository as a whole; PRISMYRA_NVFP4_EXPERTS and PRISMYRA_NVFP4_CALIB must point at the two files once
downloaded. PRISMYRA_DENSE_KEEP='self_attn|shared_expert' is what makes the engine convert only the
gated-delta-net projections and leave attention and the shared expert on FP8 -- the combination this checkpoint's
adapter was trained for and the one every number in this card was measured with; leaving PRISMYRA_DENSE unset
still runs correctly (attention, gated-delta-net and the shared expert all stay FP8, same as this checkpoint's FP8
sibling) but gives up the speed advantage this conversion buys on top of the NVFP4-experts-only form. This Use
block, the routed-expert conversion and the gated-delta-net-in-NVFP4 training stage are unchanged from the previous
revision -- this revision's adapter data changed, not how it is served. Optional: PRISMYRA_KV_FP8=1 stores the
attention key/value cache in FP8 instead of the engine's default precision, trading a small amount of accuracy
risk for more context/question headroom on a fixed-size card; not separately remeasured for this revision, see the
previous revision's card for what it measured with it on and off.
The starting point for this checkpoint is the sibling FP8 checkpoint's own v0.5.1 adapter (the one with the two new
long-document training families for variable tracking and fact chaining; see that checkpoint's own
card and
docs/DATA_PROVENANCE.md for the full accounting). On top of that adapter, the same gated-delta-net-in-NVFP4
quantization-aware training stage as the previous nvfp4 revision was run again, on the same data the FP8 sibling's
v0.5.1 adapter itself was trained on (so the two new long-document families are part of this stage's training data
too, not held out from it). The distillation signal used during this stage is this same checkpoint's own FP8 form
before this stage -- still this project's own model's output, not a third party's and not an earlier
research-trained checkpoint's.
This stage is seed-sensitive, the same way the previous nvfp4 revision's card already documented. Two training recipes were compared this round -- the quantization-aware training stage with and without that same self-distillation signal -- at two seeds each. Without the signal, 1 of 2 seeds passed this project's own accuracy bar; with it, both seeds passed accuracy, but of those two only one also passed the determinism bar below (the other's answers stayed free of decision flips but moved by more than this checkpoint's own FP8 form does on the same check, so it was not shipped). The recipe and seed shipped here is the one that passed both bars; a caller retraining this stage should expect to need more than one attempt and should check both bars, not accuracy alone, before trusting a result.
Long-context variable tracking (NVIDIA RULER's "vt" task shape, measured at 65,536 tokens -- see the FP8 sibling's card for the same measurement at 16,384 and 32,768 tokens, not separately remeasured on this NVFP4 checkpoint):
| context length | previous line (pre-v0.5.1 adapter) | this checkpoint |
|---|---|---|
| 65,536 tokens | 63.2% | 96.0% |
This checkpoint's score matches its FP8 sibling's own 65,536-token score (96.0%) to the decimal: the quantization-aware training stage did not cost anything on the long-context capability the sibling's v0.5.1 adapter added.
Broad non-inferiority against the previous nvfp4 revision, measured once on a newly built, previously unopened 10,000-question evaluation set, opened only after this checkpoint was already selected and frozen. Scoring: paired bootstrap (4,000 resamples, 95% interval), non-inferiority taken as the interval's lower bound exceeding -0.5 accuracy points:
| point difference vs. the previous revision | 95% interval | result | |
|---|---|---|---|
| final check (opened once, this revision's release gate) | +0.86 | [+0.37, +1.37] | passes |
This revision clears the non-inferiority bar, and its whole 95% interval sits above zero -- not just within noise of the previous revision, but ahead of it. Within this same check, the one family built specifically to probe long-document reading improved from 98.25% to 98.50% (+0.25 points). A separate, narrower check -- does this quantization-aware training stage itself move accuracy on top of the adapter it starts from, scored against this checkpoint's own pre-stage FP8 form on four held-out sets -- found every set's confidence interval either favoring this checkpoint or not significantly different; none regressed.
On this project's own production benchmark (2,057 questions) and against other models (autotrust/JEV-35B-NVFP4,
the earlier non-commercial -36l nvfp4 line): not remeasured for this revision. See the previous nvfp4 revision's
own card for those numbers, and the FP8 sibling's v0.5.1 card for the same comparisons carried forward on that
checkpoint -- this revision's own evaluation above already answers the question this revision needed to answer
(does it hold up against its immediate predecessor on a broad, independent set).
Not remeasured against the previous revision's own published figures: alternating this checkpoint against the previous revision six times, 1/16/64 questions at three context lengths (1k/4k/16k tokens), same RTX PRO 4500, every one of nine cells differed by at most 0.6% -- within this project's own 5% bar for calling something a real change, and expected, since this stage changes LoRA weight values and the gated-delta-net's quantization, not shape, depth, or any kernel path the routed-expert conversion already uses. Memory at rest on the RTX PRO 4500, this checkpoint with its recommended settings: 22.7 GiB, comfortably inside the 32 GB card and consistent with the previous revision's own reported range (21-23 GiB depending on key/value cache setting).
Determinism: reproduced from a from-scratch install of this project (pip install from the repository's main
branch) against the exact file set this repository publishes, immediately before this card was written.
Companion-invariance (does the same question asked about the same document get the same answer regardless of which
other documents share its pass) was checked against the project's own fixed audit (4,392 probability checks,
torch.equal on the full probability tensor): 4,392/4,392 bit-exact, zero decision flips. A second, separate
check flips only the paged construction flag (whether the engine routes a solo question through a shared page
pool instead of a single joined pass) on the same 8-document audit fixture: 2,424/2,448 bit-exact, zero decision
flips, largest probability move 0.0242 -- identical to this checkpoint's own pre-quantization-training FP8 form on
the same fixture. This project's own measurement on a much broader, 1,893-question set (not specific to this
checkpoint; see docs/PERFORMANCE.md)
found that the 8-document fixture's bound does not generalize: paging moves 8.08% of that broader set past the
fixture's bound and flips 1.69% of decisions, with no measurable aggregate accuracy cost. What this project
promises past the 8-document fixture is narrower than "every document agrees between the two paths": reproducibility
(same input, same launch mode, same bits) and fork isolation (a companion never moves an answer), plus aggregate
accuracy that does not measurably differ between the two -- not that any one document's exact answer agrees with
the other construction past this fixture. This checkpoint's own result on the fixture matches its FP8 sibling's and
its own pre-training-stage form; nothing in this round's work is in a position to say whether the broader,
1,893-question finding above would also hold or differ for this specific checkpoint, because that measurement was
not rerun on it.
littlemex/prismyra-decision-35b-a3b-fp8 --
this checkpoint's non-expert weights in FP8 instead of NVFP4 experts, for Ada/Hopper-class GPUs, with the same
adapter folded in before this checkpoint's own quantization-aware training stage.littlemex/prismyra-decision-qwen3.6-35b-a3b-nvfp4-36l
and the rest of the earlier checkpoint line -- trained in part on research-only data (see
docs/DATA_PROVENANCE.md). That line remains available and is not being withdrawn; this repository is a
separate line for callers who specifically need every training source to be commercially clear, not a
replacement.This checkpoint answers typed questions (booleans, fixed choice lists) about a supplied document; it does not
generate free-form text. It cannot read a document of 131,072 tokens or more: the engine this checkpoint runs on
has a fixed context budget that tops out at 65,536 tokens, and a longer document is refused outright rather than
truncated or answered partially. At a single question, this checkpoint's FP8 sibling is slower than
autotrust/JEV-35B (measured on the previous revision, not remeasured here); on the long-buried-context family
("bury7k"), that sibling trails autotrust/JEV-35B by a Holm-significant margin (also carried over, not remeasured
on this NVFP4 checkpoint specifically). The quantization-aware training stage this checkpoint ships remains
seed-sensitive (see Training): of the recipes and seeds tried this round, only one combination passed both the
accuracy and the determinism bar, the same pattern the previous nvfp4 revision's own card documented for itself.
Requires a Blackwell-class GPU (sm_120) with native NVFP4 tensor cores; verified on the RTX PRO 4500 (32 GB). Holds about 22.7 GiB at rest with this checkpoint's recommended settings, comfortably inside a 32 GB card.
See the Prismyra repository for how to cite this project.
Apache-2.0, the base model's licence, and this repository carries the base model's LICENSE file unchanged. Every
training source behind this adapter is confirmed clear for commercial use and for training a derivative model --
see docs/DATA_PROVENANCE.md for the
source-by-source accounting. Teacher-assisted gold labels come only from Kimi K3 and DeepSeek V3.2; no generated
rationale or free text from either model is reproduced in this checkpoint's weights or in this card.
PRISMYRA_EXPERTS=nvfp4 \PRISMYRA_NVFP4_EXPERTS=/path/to/nvfp4_experts.safetensors \PRISMYRA_NVFP4_CALIB=/path/to/nvfp4_calib.json \PRISMYRA_DENSE=nvfp4 \PRISMYRA_DENSE_KEEP='self_attn|shared_expert' \prismyra-serve --model /path/to/this/checkpoint/directory