Why marvy-1-14B

• Drafts the full lifecycle, not just snippets. Business analysis through validation — the artifacts and sequence real delivery teams actually work in.
• OOTB-first and implementation-grade. Tuned to favor out-of-the-box correctness and produce drafts you can review, not rewrite.
• Runs locally and privately. Merged FP16, a LoRA adapter, and GGUF quants — run it on Apple Silicon via LM Studio or Ollama, with your engagement data never leaving your machine.
• Trained on real, anonymized delivery work. ~1,958 redacted engagement artifacts (~887k tokens), with zero residual PII verified by an automated leakage scanner.
• Open and Apache-2.0. Built on Qwen2.5-14B-Instruct — inspect it, fine-tune it, and deploy it on your own terms.

📖 Full docs: USAGE.md (every runtime + OpenCode wiring) · VALIDATION.md (prove the fine-tune works) · validate.sh (one-command probe harness)

---

Quick start

Transformers

python
Copy code
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "MainStack/marvy-1-14B"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

SYSTEM = (
  "You are a senior ServiceNow delivery consultant. You produce precise, "
  "implementation-grade artifacts: business analyses, requirements, solution "
  "design documents, user stories with acceptance criteria, test cases, and "
  "validation reviews. You favor out-of-the-box capabilities, cite concrete "
  "tables/plugins/sys_ids when relevant, and write in clear professional English."
)

messages = [
  {"role": "system", "content": SYSTEM},
  {"role": "user", "content": "Write a ServiceNow user story with acceptance criteria for SLA escalation on P1 incidents."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=1024, temperature=0.4)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

vLLM

bash
Copy code
pip install vllm
vllm serve MainStack/marvy-1-14B

Ollama (via GGUF)

Use the companion repo MainStack/marvy-1-14B-GGUF:

bash
Copy code
ollama run hf.co/MainStack/marvy-1-14B-GGUF:Q4_K_M

MLX (Apple Silicon native)

bash
Copy code
pip install mlx-lm
python -m mlx_lm generate --model MainStack/marvy-1-14B \
  --system-prompt "You are a senior ServiceNow delivery consultant..." \
  --prompt "Draft the Platform Architecture section of an ITSM SDD." \
  --max-tokens 1024 --temp 0.4

LoRA-only (apply on top of the base)

If you prefer a tiny adapter (~175 MB) on top of the BF16 base, see MainStack/marvy-1-14B-lora.

---

Intended use

marvy-1-14B is designed to produce implementation-grade first drafts across the ServiceNow delivery lifecycle — accelerating the artifacts a practitioner would otherwise write from scratch, then review and refine. Built for solution architects, business analysts, technical consultants, and project managers. Typical tasks:

Recommended system prompt

markdown
Copy code
You are a senior ServiceNow delivery consultant. You produce precise, implementation-grade
artifacts: business analyses, requirements, solution design documents, user stories with
acceptance criteria, test cases, and validation reviews. You favor out-of-the-box
capabilities, cite concrete tables/plugins/sys_ids when relevant, and write in clear
professional English.

Recommended generation settings

---

Training data

The training dataset is proprietary to MainStack and is not publicly released. It is derived from confidential, anonymized client engagement artifacts. The statistics below describe the corpus for transparency; the data itself is not distributed with the model.

Privacy & redaction

• All customer/partner names → stable aliases (e.g. Customer-FIN-03, Customer-ENERGY-01).
• Emails → user@example.com; hostnames → instance.example.service-now.com; IPs → RFC 5737 range; key: value secrets → [REDACTED].
• Credential/login/VPN files excluded entirely; bulk CMDB dumps >1.5 MB excluded.
• ServiceNow sys_ids and table/plugin names preserved (instance-local, technically valuable, low risk).
• A leakage scanner asserts 0 residual emails, hostnames, or mapped real names in message content.

Split integrity

Train / val / test are split by project, so no customer appears in more than one split. The largest project is forced into train to keep eval honest:

• val projects: Customer-ENERGY-01
• test projects: Customer-CHEM-01, Customer-FININST-01

---

Training procedure

---

Evaluation

Fine-tuned vs. base — efficiency on the held-out test set

The cleanest measure of the fine-tune's value is to score the same base model twice — plain vs. with the marvy adapter — on the project-disjoint test split (252 records from two customers never seen in training/val), using per-token cross-entropy/perplexity on the assistant tokens only (prompt-masked, the same objective used in training). Lower perplexity = the model assigns higher probability to the real, human-authored delivery artifact.

Overall: perplexity 8.91 → 6.03, a 32.3% reduction on unseen customers.

The gains are largest on structured, format-heavy artifacts (inventories, requirements, stakeholder registers, stories) where the base model wanders from the expected schema; they are smaller on long-form prose (SDD sections, business analysis) where the base was already competent. This is the honest, expected shape of a domain SFT.

Notes: the test customers (Customer-CHEM-01, Customer-FININST-01) appear in neither train nor val, so this reflects generalization, not memorization. The test split happens to cover 7 of the 11 task families. An earlier MLX batch-eval reported aggregate ppl ≈ 13.1 with 2,048-token truncation; the figures above recompute per-task with full assistant-token masking, so the base-vs-marvy delta is the result of interest.

Reproduce it yourself: bash benchmark/run_benchmark.sh (see VALIDATION.md for qualitative probes too).

---

Limitations & known issues

• Text-only sources. SOWs/SDDs/workbooks in .docx/.pptx/.pdf/.xlsx are not parsed in this build. Coverage of binary-only engagements is therefore thin.
• Project concentration. ~95% of records come from ~12 data-rich projects; the long tail contributes a single case study each. Some task families (e.g. case_study, validation_critique) are smaller and may exhibit higher variance.
• Synthetic instructions. User prompts are templated paraphrases (3–5 variants per task); assistant outputs are the original human-authored artifacts.
• English-only. The corpus is English.
• Not a replacement for a consultant. Output is first-draft, implementation-grade content that requires expert review before client delivery or production use.
• No tool use / function calling fine-tune. marvy-1-14B is a text-completion specialist; agentic tool use is left to the orchestrator.
• Hallucination risk on instance-specific facts. The model will confidently invent sys_ids, plugin IDs, and table fields if asked about specifics it has not seen. Always verify against an actual ServiceNow instance.
• No safety fine-tune beyond the base. Inherits Qwen2.5-14B-Instruct safety behavior; no additional RLHF.

---

License

marvy-1-14B is dual-licensed — see LICENSING.md for the full breakdown:

The model weights are a derivative of Qwen2.5-14B-Instruct (Apache-2.0). Per Apache-2.0, the weights cannot be placed under a more restrictive license; MainStack's protection is the CC-BY-4.0 license on our own authored materials plus the mandatory NOTICE retention. See NOTICE for attribution.

Attribution

marvy-1-14B is free to use, fine-tune, and redistribute under Apache-2.0. If you use marvy-1-14B as a baseline, fine-tune it, distill from it, evaluate against it, or otherwise build on it, please credit MainStack and link back to this model:

Built on / evaluated against marvy-1-14B by MainStack — https://huggingface.co/MainStack/marvy-1-14B

Concretely, we ask that derivatives and comparisons:

• keep the NOTICE file intact (this is required by Apache-2.0 §4),
• name MainStack/marvy-1-14B in the model card, paper, or README, and
• cite the entry below.

Per Apache-2.0, you must also continue to attribute the upstream base model (Qwen2.5-14B-Instruct) — see NOTICE.

Citation

If you use marvy-1-14B (as a baseline, a starting point, or in evaluation), please cite:

bibtex
Copy code
@software{marvy_1_14b_2026,
  title     = {marvy-1-14B: An open fine-tuned model for the full ServiceNow delivery lifecycle},
  author    = {MainStack},
  year      = {2026},
  publisher = {Hugging Face},
  url        = {https://huggingface.co/MainStack/marvy-1-14B},
  note      = {Fine-tune of Qwen2.5-14B-Instruct},
  license   = {Apache-2.0}
}

@misc{qwen2.5,
  title  = {Qwen2.5: A Party of Foundation Models},
  author = {Qwen Team},
  year   = {2024},
  url    = {https://qwenlm.github.io/blog/qwen2.5/}
}

bibtex
Copy code
@software{marvy_14b_2026,
  title  = {marvy-1-14B: A ServiceNow delivery lifecycle fine-tune of Qwen2.5-14B-Instruct},
  author = {MainStack},
  year   = {2026},
  url    = {https://huggingface.co/MainStack/marvy-1-14B},
  license= {Apache-2.0}
}

@misc{qwen2.5,
  title  = {Qwen2.5: A Party of Foundation Models},
  author = {Qwen Team},
  year   = {2024},
  url    = {https://qwenlm.github.io/blog/qwen2.5/}
}

Acknowledgements

• Qwen team at Alibaba Cloud for the Qwen2.5 family.
• Apple MLX team for mlx and mlx-lm, enabling native Apple Silicon training.
• Hugging Face for hosting and the surrounding ecosystem.