GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
littlemex
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
littlemex
Model Tree
Qwen/Qwen3.6-35B-A3B-FP8
Input Modalities
Output Modalities
Supported Functionality
bash
pip install "prismyra[server,fast] @ git+https://github.com/littlemex/Prismyra@v0.2.2"python
from prismyra import Prismyra, Boolean engine = Prismyra("littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-32l")result = engine.ask(document, [Boolean(id="q", prompt="Is this a two-sided agreement?")])bash
prismyra-serve --model littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-32l --require-kernels--require-kernels refuses to start rather than serve at a fraction of the speed. Prismyra's fused kernels are
matched against the checkpoint by per-layer tensor shape, not by layer count, so a checkpoint with fewer layers than
the base model still gets every kernel the base model gets -- there is no flag to set for this.
Five sets, served by Prismyra on one L40S: RACE (150 articles, 579 short-passage questions), BoolQ (400), bury7k and bury10k (the same RACE questions buried in about 7,000 and 10,000 tokens of unrelated articles, 157 questions each), and Kev (a 764-question transfer test built from five other datasets). The bar for each set is the better-scoring of two other decision models, Decider-4B and Lux-9B; which one wins changes by set.
| set | this checkpoint, perm 1 | this checkpoint, perm 4 | bar | which competitor | 40-layer, perm 1 | 36-layer, perm 1 |
|---|---|---|---|---|---|---|
| RACE | 95.51 | 96.03 | 93.78 | Decider-4B | 95.85 | 96.20 |
| BoolQ | 89.25 | 89.25 | 89.50 | Decider-4B | 90.25 | 90.25 |
| bury7k | 94.90 | 95.54 | 91.08 | Decider-4B | 93.63 | 94.27 |
| bury10k | 95.54 |
This checkpoint misses the BoolQ bar by one question (357 against 358 of 400). It ties the Kev bar exactly (633
of 764 both) under perm 1 and clears it under perm 4 (84.29). It clears RACE comfortably under both orderings,
and it is ahead of both the 40- and 36-layer checkpoints on the two long-context sets (150 correct against 148 and
145 on the 10,000-token version). perm 4 -- the average of the four cyclic rotations of each Choice question's
options, which adds branch passes but no second reading of the document -- moves bury10k by more than 2.5 points in
either direction depending on the checkpoint, which is the size of ordering noise on that set.
On JevBench, a 231-question benchmark none of these checkpoints trained on, this checkpoint scores 78.35% against the untrained base's 77.06% -- a sign test finds no significant difference (p = 0.795). The 40- and 36-layer checkpoints score 87.45% and 87.88% there and are not significantly different from each other. Read the non-significant result as "no difference detected in this test," not as proof the adapter's effect at 32 layers has vanished -- but it is a real disagreement with the five-set table above, where this checkpoint is close to or ahead of the two longer ones on four of five sets, and it is the reason this checkpoint is not the default. Full numbers, and a comparison of all three checkpoints against four general-purpose LLMs on the same five sets, are in the training recipe's README.
Measured directly from Python (no network), at the engine's default request width (group=32), against the
untrained base and the other two published checkpoints, same script and GPU, on the same ~5,300-token document
repeated across calls:
| checkpoint | 1 question | 16 questions | 64 questions |
|---|---|---|---|
| base, untrained, 40 layers | 276.8 ms | 408.7 ms | 733.4 ms |
| 40-layer | 269.9 ms | 399.4 ms | 716.8 ms |
| 36-layer | 244.3 ms | 362.1 ms | 650.3 ms |
| this checkpoint (32-layer) | 218.0 ms | 324.5 ms | 584.7 ms |
This project publishes four repositories:
littlemex/prismyra-decision-lora-qwen3.6-35b-a3b
-- the LoRA adapter for the full 40-layer base model, trained separately from this checkpoint's own LoRA.littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-40l
-- all 40 layers, that adapter folded in. The teacher this checkpoint was distilled from.littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-36l
-- 36 layers, trained the same way as this checkpoint. The one recommended by default.littlemex/prismyra-decision-qwen3.6-35b-a3b-fp8-32l -- this repository.Apache-2.0, the base model's licence, and this repository carries the base model's LICENSE file unchanged. Some of
the training sources behind this checkpoint's LoRA carry their own terms -- RACE and SciQ are distributed for
non-commercial research use -- so check those terms before commercial use of this checkpoint.
| 92.99 |
| 91.08 |
| Decider-4B |
| 94.27 |
| 92.36 |
| Kev | 82.85 | 84.29 | 82.85 | Lux-9B | 85.47 | 85.73 |