GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
littlemex
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
littlemex
Model Tree
Qwen/Qwen3.6-35B-A3B-FP8
Input Modalities
Output Modalities
Supported Functionality
bash
pip install "prismyra[server,fast] @ git+https://github.com/littlemex/Prismyra@v0.2.2"python
from prismyra import Prismyra, Boolean engine = Prismyra("littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-40l")result = engine.ask(document, [Boolean(id="q", prompt="Is this a two-sided agreement?")])bash
prismyra-serve --model littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-40l --require-kernels--require-kernels refuses to start rather than serve at a fraction of the speed. Prismyra's fused kernels are
matched against the checkpoint by per-layer tensor shape, not by layer count, so they apply to this checkpoint the
same way they apply to the base model.
Five sets, each checkpoint served by Prismyra on one L40S: RACE (150 articles, 579 short-passage questions), BoolQ (400), bury7k and bury10k (the same RACE questions buried in about 7,000 and 10,000 tokens of unrelated articles, 157 questions each), and Kev (a 764-question transfer test built from five other datasets, of which 228 questions come from three families this model never trained on in any form). The bar for each set is the better-scoring of two other decision models, Decider-4B and Lux-9B; which one wins changes by set.
| set | this checkpoint | bar | which competitor |
|---|---|---|---|
| RACE | 95.85 | 93.78 | Decider-4B |
| BoolQ | 90.25 | 89.50 | Decider-4B |
| bury7k | 93.63 | 91.08 | Decider-4B |
| bury10k | 94.27 | 91.08 | Decider-4B |
| Kev | 85.47 | 82.85 | Lux-9B |
The BoolQ margin is 3 questions out of 400 (361 against 358) and is within a single evaluation run's noise; read it
as a tie with the bar, not a clear win. The other four sets clear their bar by a wider margin. Options in every
Choice question here are read in the order the source data gives them (perm 1); a companion checkpoint in this
family (36 layers) was additionally scored with the four cyclic rotations of each Choice averaged (perm 4) and
moved by 0.2 to 1.9 points per set under that averaging, which is the size of run-to-run noise on these sets, not
evidence that one ordering is correct.
This checkpoint is also measured against JevBench (231 questions none of the checkpoints in this family trained on) and against four general-purpose LLMs on the same five sets above, in the training recipe's README.
Two conditions, both measured directly from Python (no network), at the engine's default request width
(group=32). Against the untrained base and the other two published checkpoints, same script and GPU, on the same
~5,300-token document repeated across calls:
| checkpoint | 1 question | 16 questions | 64 questions |
|---|---|---|---|
| base, untrained, 40 layers | 276.8 ms | 408.7 ms | 733.4 ms |
| this checkpoint (40-layer) | 269.9 ms | 399.4 ms | 716.8 ms |
| 36-layer | 244.3 ms | 362.1 ms | 650.3 ms |
| 32-layer | 218.0 ms | 324.5 ms | 584.7 ms |
Folding the adapter in costs nothing measurable against the untrained base at the same depth. The 36-layer checkpoint is also measured, same day, against two other decision models over their own HTTP servers -- see its model card for that table.
This project publishes four repositories:
littlemex/prismyra-decision-lora-qwen3.6-35b-a3b
-- the LoRA adapter alone, trained against this checkpoint's base weights.littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-40l -- this repository. All 40 layers, that adapter folded into
the FP8 weights.littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-36l
-- 36 layers, trained with a fresh LoRA distilled from this checkpoint's own output distribution. Faster (above)
and scores 2 questions above this checkpoint on Kev (655 against 653 of 764) -- not more accurate, evidence that
the four fewer layers did not cost accuracy on that set.littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-32l
-- 32 layers, trained the same way. Faster again, and ahead of both longer checkpoints on the two long-context sets
above, but the one of the three that misses the BoolQ bar (by a single question).The 36-layer checkpoint is the one recommended by default; this one is the reference it was distilled from.
Apache-2.0, the base model's licence, and this repository carries the base model's LICENSE file unchanged. Some of
the adapter's training sources carry their own terms -- RACE and SciQ are distributed for non-commercial research use
-- so check those terms before commercial use of this checkpoint.
from prismyra import Prismyra, Boolean engine = Prismyra("littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-40l")result = engine.ask(document, [Boolean(id="q", prompt="Is this a two-sided agreement?")])