Intended use
- Automated security code review / SAST as part of a scanner that supplies per-function
context and a candidate rule set.
- Languages: Python, JavaScript/TypeScript, Java, Go, Ruby, PHP, C#, and C/C++
(memory-safety classes).
Out of scope: standalone chat, non-security code tasks, dependency-CVE (SCA) analysis,
and runtime/config issues that are not visible in source.
Vulnerability classes detected
Injection (SQL, command, code/eval), insecure deserialization (RCE), path traversal,
SSRF, XXE, XSS, open redirect, broken access control, IDOR / broken object-level
authorization (BOLA), broken authentication (incl. weak password-reset tokens), CSRF,
sensitive-data exposure, and C/C++ memory-safety (buffer overflow, UAF, integer overflow).
Evaluation
Held-out test set (1,887 examples, served FP8 via vLLM):
Table with columns: Metric, Value| Metric | Value |
|---|
| Precision | 41.0% |
| Recall | 40.9% |
| F1 | 40.9% |
| False-positive rate (clean code) | 0.7% |
| JSON-schema compliance | 98.8% |
The very low false-positive rate (0.7%) is the headline characteristic — the model is
calibrated to stay quiet on clean code rather than over-flag, which is what makes it usable
in an automated pipeline. Recall (~41%) is per-function single-pass; scanner-level recall is
higher because functions are reviewed with call-graph context and candidate rules.
Per-language recall (test set): C# 65%, Rust 53%, Ruby 52%, JS 45%, Python 44%, Java 43%,
TypeScript 42%, Go 40%, C++ 39%, C 32%.
Qualitative check with the Foil scanner against DVNA (OWASP Top-10 reference app): the
model identified the documented data-flow and access-control vulns — SQL/command/code
injection, XXE (RCE), insecure deserialization (RCE), broken access control, IDOR/BOLA
(with ownership-fix reasoning), the A2 MD5-of-login reset-token logic flaw, and unvalidated
redirect.
Known limitations. Reflected XSS whose sink lives in a separate view template
(.ejs/.hbs unescaped output) can be missed when only the handler is in context — supply
template context for full coverage. The model is thorough and may surface advisory findings
(rate-limiting, headers) beyond a strict vuln set; triage accordingly. Dependency-CVE (A9),
runtime misconfiguration (A6), and logging (A10) are out of scope for source-only review.
Serving (vLLM, OpenAI-compatible)
This is a compressed-tensors FP8 checkpoint (FP8_DYNAMIC) — vLLM auto-detects the
quantization from the checkpoint, so no --quantization flag is needed:
vllm serve vitorallo/securereview-coder-32B-FP8 \ --served-model-name securereview \ --max-model-len 32768
The model emits a JSON object:
{"findings": [{"severity", "category", "line", "code", "description", "recommendation", "confidence", "cwe_id"}]}
. For best results with this
fine-tune,
do not force guided/constrained decoding — it was trained on a fixed JSON
schema and grammar-constrained decoding can degrade output (over-flagging / missed classes).
Quantization recipe: FP8_DYNAMIC (per-channel static weight scales + per-token dynamic
activation scales), lm_head kept in higher precision.
Citation / attribution
Built on Qwen2.5-Coder. Please cite the base model:
@article{hui2024qwen2coder, title={Qwen2.5-Coder Technical Report}, author={Qwen Team}, year={2024}}