Research focus
Table with columns: Area, Intended workflow| Area | Intended workflow |
|---|
| iOS system security | Interpret Apple advisories about platform components, kernel issues, WebKit, and system services. |
| CVE research across services | Extract the product, documented impact, and remediation guidance from sourced vulnerability records. |
| Penetration-testing knowledge | Explain techniques and compare choices using public security research material. |
| Evidence-aware answers | Cite supplied sources, distinguish a release containing a fix from the affected-version range, and identify missing evidence. |
These are the goals of this research build. Its effectiveness will be assessed using the evaluation materials below.
Downloads
Use this model with local apps
The instructions below require the published Q4_K_M GGUF. Check Downloads above for availability.
Hugging Face fills the Use this model menu automatically after it recognizes a compatible model file. Enable your preferred apps in Local Apps settings; the choices are personal to each viewer. A GGUF tag by itself does not activate the menu entries. How the menu works.
Table with columns: App, How to use the released GGUF| App | How to use the released GGUF |
|---|
| LM Studio | Choose Use this model → LM Studio, or search for this repository in Discover. Setup below. |
| Ollama | Choose Use this model → Ollama, or run the command below. Hugging Face guide. |
| llama.cpp | Choose Use this model → llama.cpp, or use the server command below. Hugging Face guide. |
| Jan | Choose Use this model → Jan after installing Jan and enabling it in Local Apps settings. Local app guide. |
Ollama — after the GGUF is available:
ollama run hf.co/Volpidash/deepseek-r1-14b-security:Q4_K_M
llama.cpp — after the GGUF is available:
llama serve -hf Volpidash/deepseek-r1-14b-security:Q4_K_M -c 4096
These are GGUF loading routes, not separate compatibility or quality benchmark results. The LoRA adapter alone is not a standalone model for these apps.
Run locally with LM Studio
Once the GGUF is published:
- Open Discover in LM Studio and search for
Volpidash/deepseek-r1-14b-security, or paste this repository's URL. LM Studio download guide.
- Choose
deepseek-r1-14b-security-Q4_K_M.gguf and load it.
- Start with a 4,096-token context. Use GPU offload within your available VRAM; increase context only after checking memory usage.
- Keep the model's included DeepSeek chat template. Start with temperature 0.6 and top-p 0.95; put task instructions in the user message, following the base model's recommendations.
The 13.63 GiB result above is a training-step measurement on one RX 6800 XT configuration. GGUF inference has a different memory profile that depends on context length and offloading; file size will be recorded after export. The fine-tuning sequence length was 768 tokens, so longer-context research workflows need separate evaluation.
Example research prompts
Read an iOS advisory
Using only the dated Apple advisory excerpts below, list each CVE,the affected component, the documented impact, and the release containingthe fix. Cite the source URL for each item. Mark any affected-versionrange that the source does not establish. [Paste advisory excerpts and URLs here]
Review another service
Review this vendor advisory and CISA KEV record for the service I aminvestigating. Summarize the documented vulnerability and required action.Separate confirmed facts from details that need verification, includingthe exact affected versions. [Paste the service version, advisory, and KEV record here]
For questions such as “all publicly known CVEs for iOS 27,” provide current, dated advisories or connect a retrieval tool in your application. A fine-tune has a fixed knowledge snapshot and cannot establish a complete, current CVE inventory from memory. This project has not yet validated native tool calling or autonomous research workflows.
Training data
The retained split combines source-based advisory tasks with security question-and-answer examples:
The preparation process grouped Apple records by advisory and the remaining records by CVE or question, then removed training records sharing CVE IDs with evaluation. The automated audit found zero shared groups and zero shared CVE IDs. Entire examples that exceeded the token budget were excluded rather than truncating their answers.
Apple and CISA tasks use source excerpts. Pentesting Explanations is third-party synthetic data; its answers have not been independently fact-checked. CISA KEV represents known exploited vulnerabilities, which is a subset of publicly disclosed vulnerabilities.
Reproducibility: data manifest · split and formatting audit.
Measured data coverage
Table with columns: Coverage measure, Training, Held out| Coverage measure | Training | Held out |
|---|
| Distinct CVE IDs mentioned | 1190 | 93 |
| iOS 27 / 27.x advisory examples | 326 | 0 |
| Kernel component examples | 150 | 1 |
| WebKit component examples | 201 | 11 |
| CISA vendor labels | 158 | 29 |
| CISA vendor/product pairs | 264 | 36 |
Coverage gap: the held-out advisory set has no iOS 27 examples, so this run does not measure generalization to unseen iOS 27 advisories. Apple tasks primarily cover advisory interpretation, not kernel source-code auditing. Counts describe data, not model competence or completeness of public CVE coverage.
Full coverage report · Counts and corpus hashes
Benchmarks
Paired security benchmark
Table with columns: Security suite, Base NF4, Fine-tune NF4, Change| Security suite | Base NF4 | Fine-tune NF4 | Change |
|---|
| Held-out Pentesting Explanations | 86.4% (152/176; 95% CI 80.5%–90.7%) | 86.4% (152/176; 95% CI 80.5%–90.7%) | +0.0 pp |
| SecEval adapted subset | 86.5% (173/200; 95% CI 81.1%–90.6%) | 86.5% (173/200; 95% CI 81.1%–90.6%) | +0.0 pp |
Full measured results, topic breakdowns, and predictions.
Choices are balanced A/B/C/D (25% guessing expectation). The comparison uses forced next-letter ranking with reasoning closed, not generated answers. It does not measure normal R1 reasoning or the Q4_K_M export. SecEval is an adapted subset, not an official leaderboard run.
Protocol and limitations · Frozen selection and hashes
Published base-model results
These scores were reported by DeepSeek for DeepSeek-R1-Distill-Qwen-14B. The general-purpose suites below have not been run on this security fine-tune or its Q4_K_M export.
Table with columns: Benchmark, Metric, Official base model, This fine-tune| Benchmark | Metric | Official base model | This fine-tune |
|---|
| AIME 2024 | pass@1 | 69.7% | Not measured |
| MATH-500 | pass@1 | 93.9% | Not measured |
| GPQA Diamond | pass@1 | 59.1% | Not measured |
| LiveCodeBench | pass@1 | 53.1% | Not measured |
| Codeforces | Rating | 1,481 | Not measured |
Source: DeepSeek's official distilled-model evaluation. DeepSeek reports a generation limit of 32,768 tokens and, for sampling-based evaluations, temperature 0.6, top-p 0.95, and 64 responses per query. Those conditions differ from this project's local evaluation. Fine-tuning and quantization can change the scores.
Measured local hardware check
Table with columns: Measurement, Result, Scope| Measurement | Result | Scope |
|---|
| Peak allocated GPU memory | 13.63 GiB | One NF4 + LoRA training step on RX 6800 XT |
| First training-step duration | 97.15 seconds | Synthetic 768-token sequence; includes first-step overhead |
| Gradient validation | 384 finite gradient tensors | Adapter backward pass completed |
| Dataset split checks | 0 shared groups / 0 shared CVEs | Training versus held-out records |
These are capacity and integrity checks. The first-step duration is not an inference throughput benchmark or a steady-state training speed estimate. Hardware test record · Dataset audit.
Local security evaluation
Table with columns: Check, Result| Check | Result |
|---|
| Base held-out loss | 1.5157 |
| Fine-tuned held-out loss | 0.8063 |
| GGUF compatibility | Passed CPU load and four-token generation in llama.cpp |
Base metrics · Fine-tuned metrics · Before answers · After answers · GGUF check
The evaluation plan compares the unmodified base with the adapter on the same 64-record subset of the held-out split. Generation checks sample Apple advisories, CISA records, and penetration-testing questions, and include an iOS 27 freshness question. The full held-out split contains 264 records.
Generated-answer files record whether the model finished within its token budget and distinguish final answers from reasoning text. Lower loss, mentioning the expected CVE, or successfully loading a GGUF does not by itself establish factual accuracy or improved research ability. Review the generated answers against their supplied sources.
Generation probes are currently deferred to prioritize training and export. Their files may be partial or empty and must not be treated as a completed before/after comparison. The separate paired security benchmark remains in the release pipeline.
Training recipe
Table with columns: Setting, Value| Setting | Value |
|---|
| Base revision | 1df8507178afcc1bef68cd8c393f61a886323761 |
| Weight loading | NF4, double quantization; FP16 compute |
| LoRA | Rank 8, alpha 16, dropout 0.05 |
| Target layers | q_proj, k_proj, v_proj, o_proj |
| Trainable parameters | 12,582,912 — approximately 0.085% |
| Planned epochs | 1 |
| Batch / accumulation | 1 / 8 |
| Sequence length | 768 tokens |
| Learning rate | 1e-5, cosine schedule, approximately 5% warmup |
| Optimizer | AdamW |
| Memory reduction | Gradient checkpointing |
| Loss | Answer tokens only; prompt and reasoning delimiters masked |
The original DeepSeek inference template is retained. The dataset supplies final answers, so this run does not supervise new reasoning traces. Training uses a local Windows ROCm environment; the measured hardware result applies to that specific configuration.
License and attribution
Model artifacts follow the base model's MIT license; see LICENSE and the DeepSeek R1 model card. The base model is distilled from DeepSeek R1 using the Qwen2.5-14B architecture.
Credit to DeepSeek, Qwen, Apple, CISA, and Christopher Dickinson / theelderemo. Pentesting Explanations is published under Apache-2.0; the CISA feed is published under CC0. Source material retains its respective terms and attribution.
This is a community research project maintained by Volpidash, independently of the organizations above.
Page updated 2026-11-09 20:07 UTC. Release status is a publication snapshot.