Layers
Table with columns: Part, Format| Part | Format |
|---|
| MLP, lm_head | NVFP4 (Local-Hessian) |
| Full and linear attention | FP8 |
| Embeddings, vision | BF16 |
Calibration: 512 samples from 7 Nemotron post-training v3 sets.
Base revision 108f0afb.
Quality against BF16
Thinking off, temperature 0. Both measured by me with the same harness and vLLM build.
Table with columns: GPQA Diamond (198), IFEval (200) | GPQA Diamond (198) | IFEval (200) |
|---|
| BF16 | 81.8% | 85.5% |
| This model | 80.3% | 83.0% |
Speed on one DGX Spark
One user, vLLM 0.33.0 nightly. Size 20 GB (BF16 is 55 GB).
Table with columns: Context, Decode| Context | Decode |
|---|
| short | 13.2 tok/s |
| 32K | 11.8 tok/s |
| 64K | 10.7 tok/s |
| 128K | 9.1 tok/s |
128K prompt read: 82 s. Second turn on the same 128K: 1.1 s with prefix caching.
Lowest free memory during the 128K run: 65 GiB.
Run
vllm serve atakans/Arete-27B-NVFP4 --max-model-len 262144 --reasoning-parser qwen3 \ --enable-auto-tool-choice --tool-call-parser qwen3_coder --enable-prefix-caching
Needs Blackwell (NVFP4). No MTP weights in Arete, so no draft model.