Evaluation (System 1 /v1/decide)
Same harness and states as the bf16 model, compared one decision at a time:
Table with columns: NVFP4 (this), FP8, bf16 | NVFP4 (this) | FP8 | bf16 |
|---|
| Top choice agrees with bf16 | 90.5% (266/294) | 97.1% | reference |
| Largest probability change on any option | 0.164 | 0.061 | none |
| Sanity scenarios | 6/6 | 6/6 | 6/6 |
| VRAM in use, RTX PRO 6000 | 37.6 GB | 42.6 GB | about 78 GB |
| 24-decision tick, p50 | 1130 ms | 1152 ms | 1253 ms |
This is a narrow test of the decision head. System 2 (free-text generation) quality was not separately
benchmarked.
Serving
Serve it like the original (vLLM with --quantization compressed-tensors). serve_decide.py is
included. FlashInfer compiles its FP4 kernels on first use, so set CUDA_HOME to your CUDA toolkit.
License
Apache-2.0, as with the original. This is a modified (quantized) version of autotrust/JEV-27B-VL,
which is itself based on Qwen/Qwen3.8-27B. LICENSE is included unchanged.