Training data
107,665 PubMed abstracts (two MeSH-term pools: bulk lipid biochemistry/lipidomics/membrane
biophysics, and Raman/IR/vibrational spectroscopy vocabulary), trained on 64,000 of them
(60.6% of one epoch, step-capped to a fixed compute budget).
Results (measured, not estimated)
Held-out perplexity on the same 2,000-example set, base vs. fine-tuned:
Table with columns: Perplexity | Perplexity |
|---|
| Base (microsoft/Phi-3.5-mini-instruct) | 4.911 |
| Fine-tuned (this adapter) | 3.955 |
| Improvement | 19.5% lower |
Citation-grounding was separately verified to survive the fine-tune: 0/3 hallucinated evidence
citations across real domain questions plus a deliberate out-of-domain probe (details in the
main repo's docs/solutions.md).
Usage
This model is designed to answer only through a grounded-retrieval + citation-checking
pipeline — it should not be used for open-ended generation without retrieved evidence attached.
See scripts/build_evidence.py
and scripts/generate_finetuned_answer.py
for the reference pipeline.
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3.5-mini-instruct")
model = PeftModel.from_pretrained(base, "srikarjy025/lipidos-phi3-domain-adapt")
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3.5-mini-instruct")
A standalone merged version (no peft required) is available at
srikarjy025/lipidos-phi3-domain-adapt-merged.
This llama model was trained 2x faster with Unsloth.