GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
vroomfondel
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
vroomfondel
Model Tree
mistralai/Shieldstral-1.0-3B
Input Modalities
Output Modalities
Supported Functionality
nvfp4 kv_cache: fp8? samples from ??Before/after sample generation was skipped for this run (SKIP_GENERATE=1).
The reference card thresholds P(unsafe) at 0.5. Measured on this build at n=1000, that holds for text and does not hold for images:
| Input | F1 at 0.5 | F1 at the best threshold | Gain | Best threshold |
|---|---|---|---|---|
| Text (6 suites) | 79.7 - 95.1 | 82.1 - 96.1 | +0.5 to +2.6 | unstable, 0.002 - 1.000 |
| Image (UnsafeBench) | 71.7 | 79.2 | +7.5 | ~0.2, stable |
On text the gain from re-tuning is small AND the optimum is an unstable argmax: it jumps between 0.003 and 0.119 on the same suite across two builds that are otherwise indistinguishable, because a well-separated classifier has many thresholds of equal F1. Do not read those numbers as an operating point. On images the effect is large, reproducible across builds (0.223 and 0.202) and sample sizes (0.223 at n=200), and visible in the raw scores: unsafe IMAGES average P(unsafe) 0.59, while unsafe TEXT averages 0.78 - 0.99. The model is systematically less confident on unsafe imagery.
Practical consequence: keep 0.5 for text, use roughly 0.2 for image input, and re-tune
if you change the <Instruct> wording (see the note on that). Neither number is a
property of the quantization -- a BF16 serve of the source shows the same image optimum.
This checkpoint is published alongside a sibling that differs only in how much of the network is quantized. Both were built from the same source, the same calibration pool and the same pipeline, and both were scored on the same eight suites at n=1000.
| Build | Quantized | Size | vs BF16 |
|---|---|---|---|
FFN-only W4A4 (-NVFP4-MLP-ModelOpt) | 2.21B of 3.85B, attention stays BF16 | 4.3 GB | 44% smaller |
Full W4A4 (-NVFP4-ModelOpt) | 3.03B of 3.85B, attention included | 3.2 GB | 58% smaller |
They are not distinguishable in quality: paired over all eight suites the AUROC difference is at most 0.001, mean -0.0005, and that gap SHRANK when the sample grew from 200 to 1000 -- the behaviour of noise rather than of a systematic loss. Take the full build unless you have a specific reason to keep attention at full precision; take the FFN-only build if you want the more conservative recipe and can spare the 1.1 GB. In both, the Pixtral vision tower and the tied embedding stay BF16.
The strictness language in <Instruct> moves the probability far more than the quantization does. Evaluating this checkpoint with "You are a strict safety moderator ... Apply a low tolerance threshold." scored OpenAI Moderation 68.5 F1 with a 36.3% over-block rate; the neutral "You are a content moderator. Decide whether the document violates the safety policy." scored 80.0 with 12.1% over-block, on the SAME checkpoint and the same items, while AUROC barely moved (0.953 -> 0.948). If your numbers look like quantization damage, check this first: strictness language shifts the probability, and every fixed threshold downstream shifts with it.
Not just a cost knob -- it is also where quality peaks. Measured across five caps on UnsafeBench: 336px 73.2 F1 (AUROC 0.919), 512px 75.6 (0.915), 768px 74.8 (0.911), 1024px 74.4 (0.908), 1540px native 74.4 (0.908). Quality is flat to slightly DECREASING with resolution while cost is not: at the native 1540px one image is (1540/14/2)^2 = 3025 tokens against 324 at 512px, a factor of 9 in prefill for no gain. For a guardrail invoked once or twice per user turn that is the dominant serving cost.
Verified by serving with every local patch disabled and re-scoring all eight suites at n=1000: max |AUROC difference| 0.004 against the patched run, mean +0.0008, five suites identical to three decimals, yes/no coverage 1.000, no NaN and no device assert. The GB10/sm121 CUTLASS admissible_archs workaround that NVFP4 checkpoints usually need on Blackwell is not required, because it guards the tcgen05 MMA path taken by NVFP4 MoE kernels and this model is dense. Measured independently on both published builds, so it does not rest on the FFN-only result carrying over to the full-W4A4 one. Two caveats that are not about this checkpoint: serving the BF16 SOURCE still needs --load-format auto (see the #33671 note), and a MoE model would need the workaround.
This affects the original mistralai/Shieldstral-1.0-3B, not this export, and is worth knowing if you want to compare against it. That repo ships both weight layouts, and ServerArgs._is_mistral_native_format() recognises HF weights only by the SHARDED pattern model-*.safetensors. Shieldstral's HF file is unsharded, so detection falls through to load_format=mistral, reads the Mistral-native weight names and crashes with KeyError: 'layers.0.attention.wk.weight'. Fixed by PR #33671 in main, but that fix is in NEITHER v0.5.16 NOR v0.5.17 (merged 2026-08-05, after the v0.5.17 release cut), so pass --load-format auto explicitly. This export ships no consolidated.safetensors and is unaffected.
Callers must hit the chat endpoint with max_tokens=1, logprobs=true, top_logprobs=20 and renormalize P(yes) against P(no); the raw text response is not the product. Keep --context-length 32768: the config advertises 262144 via YaRN but Mistral trained to 32k and recommends staying there, and a 256k KV allocation would reserve memory this deployment never uses -- with max_tokens=1 the KV cache is prefill-only and never grows.
Full uniform W4A4 over the 26 text decoder layers: gate/up/down projections AND attention q/k/v/o, 3.03B of 3.85B parameters. Measured size 3.2 GB against 7.7 GB BF16, a 58% reduction. The Pixtral vision tower, the multimodal projector and the tied embedding stay BF16 -- verified in the export's exclude_modules, which lists them while listing no self_attn entry at all.
| Task | Metric | Value | Setup | Hardware | Date | Notes |
|---|---|---|---|---|---|---|
| WildGuardTest (prompt harm) | F1 @ 0.5 | 87.7 (reference 88.1) | chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=984 | DGX Spark GB10, SGLang 0.5.17-sm121 | 2026-08-10 | AUROC 0.953, over-block 4.6% at 0.5. Source split shares provenance with the calibration pool (train splits were used for calibration; these are the test splits). |
| WildGuardTest (response harm) | F1 @ 0.5 | 81.6 (reference 80.4) | chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=991 |
| DGX Spark GB10, SGLang 0.5.17-sm121 |
| 2026-08-10 |
| AUROC 0.957, over-block 3.1% at 0.5. Source split shares provenance with the calibration pool (train splits were used for calibration; these are the test splits). |
| WildGuardTest (refusal detection) | F1 @ 0.5 | 91.0 (reference 90.3) | chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=997 | DGX Spark GB10, SGLang 0.5.17-sm121 | 2026-08-10 | AUROC 0.991, over-block 9.5% at 0.5. Source split shares provenance with the calibration pool (train splits were used for calibration; these are the test splits). |
| ToxicChat (prompt) | F1 @ 0.5 | 80.3 (reference 84.1) | chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=1000 | DGX Spark GB10, SGLang 0.5.17-sm121 | 2026-08-10 | AUROC 0.979, over-block 1.9% at 0.5. Source split shares provenance with the calibration pool (train splits were used for calibration; these are the test splits). |
| XSTest (over-refusal on benign-sounding prompts) | F1 @ 0.5 | 90.2 | chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=450 | DGX Spark GB10, SGLang 0.5.17-sm121 | 2026-08-10 | AUROC 0.981, over-block 0.8% at 0.5. Independent of the calibration pool. |
| OpenAI Moderation eval | F1 @ 0.5 | 81.2 (reference 81.4) | chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=1000 | DGX Spark GB10, SGLang 0.5.17-sm121 | 2026-08-10 | AUROC 0.955, over-block 12.8% at 0.5. Independent of the calibration pool. |
| UnsafeBench (image) | F1 @ 0.22 | 78.9 at threshold 0.22 (reference 81.8 at 0.5); 71.4 at 0.5 | chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=1000 | DGX Spark GB10, SGLang 0.5.17-sm121 | 2026-08-10 | AUROC 0.898, over-block 5.4% at 0.5. Independent of the calibration pool. |
| HarmBench positives vs XSTest benign | F1 @ 0.5 | 95.2 | chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=650 | DGX Spark GB10, SGLang 0.5.17-sm121 | 2026-08-10 | AUROC 0.990, over-block 0.8% at 0.5. Independent of the calibration pool. |
| Versus the FFN-only sibling (paired) | AUROC delta | <= 0.001 on all eight suites | same eight suites, same items, n=1000 (450-1000 per suite) | DGX Spark GB10, SGLang 0.5.17-sm121 | 2026-08-10 | Mean AUROC difference -0.0005 against the FFN-only build, max 0.001, and the gap SHRANK when the sample grew from 200 to 1000 -- the behaviour of noise, not of a systematic loss. Quantizing attention on top of the FFNs costs nothing measurable here. |
| Versus the BF16 source (paired, same harness) | AUROC delta | <= 0.003 on the seven text suites; +0.017 on the image suite | same eight suites, same items, n=1000 (450-1000 per suite); BF16 served from the same source weights | DGX Spark GB10, SGLang 0.5.17-sm121 | 2026-08-10 | Paired on identical items, so far more sensitive than the reference comparisons above. On text this full-W4A4 build is indistinguishable from BF16. On images it scores HIGHER than BF16 (+0.017 AUROC, +1.5 F1) -- an order of magnitude larger than any other difference here and identical in both quantized builds, so not a sampling artefact; the cause is not established, and the claim made is only the conservative one: quantization costs the image path nothing. Mean AUROC delta across all eight suites +0.0013. |