GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
littlemex
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
littlemex
Model Tree
Qwen/Qwen3.6-35B-A3B-FP8
Input Modalities
Output Modalities
Supported Functionality
bash
pip install "prismyra[server,fast] @ git+https://github.com/littlemex/Prismyra@main"python
from prismyra import Prismyra, Boolean engine = Prismyra("littlemex/prismyra-decision-35b-a3b-fp8")result = engine.ask(document, [Boolean(id="q", prompt="Is this a two-sided agreement?")])bash
prismyra-serve --model littlemex/prismyra-decision-35b-a3b-fp8 --require-kernels--require-kernels refuses to start rather than serve at a fraction of the speed.
The adapter targets the same module set as the previous revision (attention and gated-delta-net projections, the
shared expert) at rank 64, and starts from the same place in the training recipe (the unmodified base model, never
an earlier checkpoint's trained weights). This revision adds two new, programmatically generated training families
aimed at variable-tracking and multi-step fact-chaining over long documents -- a capability gap the previous
revision's evaluation showed plainly on long-context tasks. Each row's correctness is decided mechanically by the
generator itself, not by a teacher model; no new third-party data source was introduced to build them. Everywhere
else in the adapter's data, each row's distillation signal, where used, now comes from this project's own previous
commercially-clear checkpoint (this repository's own v0.5.0 revision) rather than the clean, unmodified base model --
still this project's own model's output, never an earlier research-trained checkpoint's and never a closed,
proprietary model's. Teacher-assisted gold labels, where a dataset's own label was overridden, continue to come only
from Kimi K3 and DeepSeek V3.2. See
docs/DATA_PROVENANCE.md for the full
family-by-family source and licence accounting, including this revision's "v0.5.1" section.
Every figure below is measured through the library call (engine.ask(), paged=False), not through
prismyra-serve --batcher -- see Determinism below for what moves between the two on a single question.
Long-context variable tracking (NVIDIA RULER's "vt" task shape, measured at three context lengths; this revision's targeted improvement):
| context length | previous revision | this revision |
|---|---|---|
| 16,384 tokens | 71.2% | 97.6% |
| 32,768 tokens | 67.2% | 98.4% |
| 65,536 tokens | 63.2% | 96.0% |
Broad non-inferiority, measured twice against the previous revision, on two independent 10,000-question evaluation sets drawn from the same broad population and each opened only once: a development-time check, then a single final check on a second, previously unopened set. Scoring: paired bootstrap (4,000 resamples, 95% interval), non-inferiority taken as the interval's lower bound exceeding -0.5 accuracy points.
| point difference vs. the previous revision | 95% interval | result | |
|---|---|---|---|
| development-time check | -0.05 | [-0.43, +0.30] | passes |
| final check (opened once, this revision's release gate) | +0.31 | [-0.11, +0.72] | passes |
This revision clears the non-inferiority bar on both checks, and on the final check its point estimate is actually ahead of the previous revision, not just within noise of it. Within the final check, the one family built specifically to probe long-document reading (a comprehension question about a passage with a long embedded document) improved from 98.75% to 99.12% (+0.38 points) -- the long-document weakness the Training section describes targeting did not regress on this independent check.
On this project's own production benchmark (the same five public-plus-held-out sets, 2,057 questions, used for every checkpoint this project publishes), this revision's pooled difference against the previous revision is -0.34 points [-1.07, +0.39] (sign test 26-33, not significant at this sample size) -- the same picture as the previous revision's own comparison against its predecessor: this benchmark's 2,057 questions are too coarse to resolve a difference the 10,000-question checks above can see.
Against other models (autotrust/JEV-27B-VL, autotrust/JEV-35B, autotrust/JEV-35B-NVFP4, and the project's
own earlier, non-commercial-trained -36l line), this card carries forward the previous revision's own measurements
rather than re-running them, because this revision's own evaluation above already answers the question this
revision needed to answer (does it hold up against its immediate predecessor); see the previous revision's card
history and
recipes/decision-lora/README.md
for those numbers, measured on the v0.5.0 line, not on this revision. On that measurement: against autotrust/JEV-35B
this project's commercially-clear line wins on pooled accuracy (+0.68 points) but loses, with Holm-corrected
significance, on the long-buried-context family ("bury7k", -5.10 points); at one question JEV-35B is faster, and at
16 and 64 questions this project's line is faster, by a widening margin. This revision has not been independently
remeasured against JEV-35B; nothing in this revision's own training targeted the bury7k family specifically, so
there is no basis in this card to expect that comparison to have moved.
Not remeasured for this revision. The previous revision's own card reports no measurable speed or memory difference against its predecessor (within 1%, under the 5% bar this project requires before calling something a real change), and this revision changes only LoRA weight values on the same module set, same rank, same quantisation shape -- not shape, depth or kernel path -- so the previous revision's numbers continue to describe this revision's cost.
Determinism: reproduced from a from-scratch install of this project (pip install from the repository's main
branch) against the exact file set this repository publishes, immediately before this card was written.
Companion-invariance (does the same question asked about the same document get the same answer regardless of which
other documents share its pass) and paged-vs-joined invariance (does a server-level caching setting, unrelated to
this checkpoint's weights, move an answer) were both checked the same way as the previous revision's card: 4,392
probability checks for companion-invariance, 2,448 for paged-vs-joined, torch.equal on the full probability
tensor, not just the chosen answer.
| companion-invariance | paged-vs-joined | |
|---|---|---|
| previous revision (baseline, same engine) | 4,320/4,392 bit-exact; 0 decision flips; largest probability move 0.0269 | 2,424/2,448 bit-exact; 0 decision flips; largest probability move 0.0392 |
| this revision | 4,320/4,392 bit-exact (same 72 non-exact cases, same breakdown by companion count); 0 decision flips; largest probability move 0.0242 | 2,424/2,448 bit-exact; 0 decision flips; largest probability move 0.0138 |
Zero decision flips in both checks, and every non-exact case this revision has is one the previous revision
already had, at a smaller magnitude of movement. This revision introduces no new determinism exception.
Companion-invariance holds regardless of how it is served (always true: a question's answer does not depend on
who else shares its pass). The paged-vs-joined column above does not: it is measured on 8 RACE documents, and a
caller serving this checkpoint with prismyra-serve --batcher (which always pages) sees a larger gap against the
library call (paged=False) on real content than that column suggests -- on this project's own 1,893-question
development set, one real HTTP request per question, 1.69% of answers flip decision (largest probability move
0.266046), with no measurable change in aggregate accuracy (+0.21 points, p=0.58). This project does not promise
that any one real document's answer agrees between --batcher and the library call beyond the RACE-document
figures above; drop --batcher when a single question's answer needs to match the library call to the bit. See
Prismyra's own docs/PERFORMANCE.md
for the full measurement.
littlemex/prismyra-decision-35b-a3b-nvfp4 --
this checkpoint's routed experts in NVFP4 instead of FP8, for Blackwell GPUs, with the same adapter folded in --
and, with its recommended settings, its gated-delta-net projections in NVFP4 too, which this FP8 repository does
not offer. As of this card, that repository's own revision history may lag this one; check its card for which
revision it currently serves.littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-36l
and its -40l/-32l siblings -- the earlier checkpoint line, trained in part on research-only data (see
docs/DATA_PROVENANCE.md). That line remains available and is not being withdrawn; this repository is a
separate line for callers who specifically need every training source to be commercially clear, not a
replacement.This checkpoint answers typed questions (booleans, fixed choice lists) about a supplied document; it does not
generate free-form text. It cannot read a document of 131,072 tokens or more: the engine this checkpoint runs on
has a fixed context budget that tops out at 65,536 tokens, and a longer document is refused outright rather than
truncated or answered partially. At a single question, this checkpoint is slower than autotrust/JEV-35B (measured
on the previous revision, not remeasured here; see Evaluation). On the long-buried-context family ("bury7k"), this
checkpoint trails autotrust/JEV-35B by a Holm-significant margin (also measured on the previous revision). Callers
who do not need every training source to be commercially clear and want the previous, non-commercial-trained line's
accuracy on long-document reading or the broad internal development set should consider that line instead (see
docs/DATA_PROVENANCE.md).
Full-weight FP8 at this depth needs about 32 GB of VRAM, which fits on one 48 GB L40S but not on a 32 GB card (see the NVFP4 sibling for a Blackwell-class 32 GB alternative). All numbers in this card were measured on L40S (sm_89); FP8 block-scaled kernels require a GPU with FP8 tensor-core support.
See the Prismyra repository for how to cite this project.
Apache-2.0, the base model's licence, and this repository carries the base model's LICENSE file unchanged. Every
training source behind this adapter, including the two families added in this revision, is confirmed clear for
commercial use and for training a derivative model -- see
docs/DATA_PROVENANCE.md for the
source-by-source accounting. Teacher-assisted gold labels come only from Kimi K3 and DeepSeek V3.2; no generated
rationale or free text from either model is reproduced in this checkpoint's weights or in this card.
from prismyra import Prismyra, Boolean engine = Prismyra("littlemex/prismyra-decision-35b-a3b-fp8")result = engine.ask(document, [Boolean(id="q", prompt="Is this a two-sided agreement?")])