Serving
Serve it with the Standard One engine (our SGLang fork with the schema-head readout; build and run commands in the
main card).
Use this repository as --model-path and its head/ as --decision-readout-path; the other arguments are the same.
Engines that load compressed-tensors FP8 read the backbone; the schema head needs the Standard One engine or the
PyTorch reference server. The reference server in StandardOne-3B-SH is written for the BF16 checkpoint; use the BF16
release with it.
Validation
Measured 2026-10-13 on one AMD Instinct MI350X with our SGLang-based decision engine (ROCm build, with the fix in the notes),
against the official Decision Index 0.3 run of the BF16 release (StandardOne-3B-SH v3) on the same requests. The sample takes 150
random requests from each of six benchmarks where the model is strong (Home appliance simulator, ContractNLI, HoVer, CLINC150, BANKING77, POP909-CL) and six where it is weak
(MuSR, WinoGrande, GSM8K, CRUXEval, ChessBench, MMLU-Pro); ContractNLI has 123 in total. Accuracy is the most probable answer against the benchmark's gold.
Table with columns: Benchmarks, Questions, BF16, FP8 (this repository), Change (points), Same answer as BF16| Benchmarks | Questions | BF16 | FP8 (this repository) | Change (points) | Same answer as BF16 |
|---|
| strong (6 benchmarks) | 4,209 | 92.68 % | 92.63 % | -0.05 | 99.31 % |
| weak (6 benchmarks) | 810 | 46.42 % | 46.79 % | +0.37 | 94.44 % |
| all | 5,019 | 85.22 % | 85.24 % | +0.02 | 98.53 % |
Answers change almost only where the model is unsure: on the strong benchmarks FP8 gives the same answer as BF16 for 99.3 % of questions, on the weak ones for 94.4 %, and accuracy stays within a fraction of a point in both. Per-benchmark numbers are in release-manifest.json.
Notes
- On AMD ROCm with the native per-token activation-quantization fallback, SGLang 0.5.20 needs a small fix when the
engine pads the token dimension (the per-token scale buffer is padded but the fallback writes unpadded rows);
without it the server fails at warmup. This release was validated on NVIDIA (CUDA), where the fallback is not used.
License
Apache-2.0, the same terms as StandardOne-3B-SH,
built on Standard One 3B and
Ministral 3 3B (Apache-2.0).