BioEvidence-LLM-1.5B API & Inference Endpoint

⚠️ Mandatory Medical Safety Disclaimer

This system is intended for biomedical research and educational use. It is not a substitute for professional medical advice, diagnosis, treatment, or clinical decision-making.

📋 Model Details

Developed by: Bhupati Talhande (Bhupati1998)
Model Type: Parameter-Efficient Fine-Tuning (PEFT) LoRA Adapter for Causal Language Modeling
Base Model: Qwen/Qwen2.5-1.5B-Instruct
Language(s): English (en)
License: MIT
GitHub Repository (Source Code): https://github.com/PRATHMESHTALHANDE/BioEvidence-LLM-4bit-QLoRA
Hugging Face Model: https://huggingface.co/Bhupati1998/BioEvidence-LLM-1.5B
Datasets: https://huggingface.co/datasets/Bhupati1998/BioEvidence-Datasets

🎯 Intended Uses & Scope

Direct Use

Biomedical Evidence Synthesis: Answering clinical queries grounded strictly on provided PubMed abstracts.
Trial Outcome Classification: Classifying study outcomes as YES (statistically supported), NO (ineffective/harmful), or MAYBE (inconclusive/ambiguous).
Verbatim Evidence Grounding: Extracting direct quotations containing exact statistical values ( $p$ -values, hazard ratios, confidence intervals).
Uncertainty & Limitation Extraction: Preserving sample size constraints, short follow-up periods, and geographic caveats.

Out-of-Scope Use

Automated clinical diagnosis without human physician oversight.
Direct prescribing of medical treatments or drug dosages.
Processing unverified medical claims without source literature context.

🚀 How to Get Started with the Model

python

import jsonimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel BASE_MODEL_NAME = "Qwen/Qwen2.5-1.5B-Instruct"ADAPTER_REPO_ID = "Bhupati1998/BioEvidence-LLM-1.5B" # 1. Load Tokenizer & Base Modeltokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_NAME, trust_remote_code=True)base_model = AutoModelForCausalLM.from_pretrained(    BASE_MODEL_NAME,    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,    device_map="auto" if torch.cuda.is_available() else None,    trust_remote_code=True,) # 2. Attach Trained LoRA Adaptermodel = PeftModel.from_pretrained(base_model, ADAPTER_REPO_ID)model.eval() # 3. Define Biomedical Question & Evidence Contextquestion = "Does statin therapy reduce 30-day cardiovascular mortality in patients with type 2 diabetes?"context = (    "BACKGROUND: Cardiovascular events represent the primary source of excess mortality in diabetic patients. "    "METHODS: In a multi-center randomized controlled trial of 1,200 diabetic adults, subjects were assigned to daily atorvastatin 20mg or placebo. "    "RESULTS: At 30 days, cardiovascular mortality was 2.8% in the atorvastatin arm versus 5.1% in the placebo arm (hazard ratio 0.54, 95% CI 0.38-0.78, p=0.002). "    "CONCLUSIONS: Statin therapy significantly reduces short-term cardiovascular mortality in diabetic adults.") # 4. Format Prompt with System Instructionmessages = [    {        "role": "system",        "content": (            "You are BioEvidence-LLM, a specialized biomedical evidence-grounded AI. "            "Analyze the biomedical question strictly based on the provided evidence context. "            "Output your answer ONLY as a valid JSON object with the keys: decision, answer, evidence, uncertainty, limitations."        ),    },    {        "role": "user",        "content": f"Evidence Context:\n{context}\n\nQuestion: {question}",    },] prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) # 5. Generate Grounded JSON Responsewith torch.no_grad():    outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.1, do_sample=False) response_text = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)print(response_text)

📊 Output Schema Contract

The model guarantees structured output adhering to a 5-field JSON contract:

json

{  "decision": "yes",  "answer": "Statin therapy significantly reduces 30-day cardiovascular mortality in patients with type 2 diabetes.",  "evidence": [    "At 30 days, cardiovascular mortality was 2.8% in the atorvastatin arm versus 5.1% in the placebo arm (hazard ratio 0.54, 95% CI 0.38-0.78, p=0.002)."  ],  "uncertainty": null,  "limitations": [    "Trial evaluated short-term 30-day follow-up duration (n=1200)."  ]}

⚙️ Training Details

Training Infrastructure & Hyperparameters

Table with columns: Hyperparameter, Value, Description
Hyperparameter	Value	Description
Compute Hardware	NVIDIA GeForce RTX 3050 Laptop GPU	4,096 MiB GDDR6 VRAM
CPU Architecture	12th Gen Intel Core i7-12650H	10 Cores, 16 Threads
Quantization Scheme	4-bit NormalFloat4 (`nf4`)	Double Quantization enabled (`bitsandbytes`)
PEFT Method	LoRA ( $r$ , Dropout )

📈 Verified MLflow Run Telemetry (`outputs/experiments/mlflow.db`)

Table with columns: Telemetry Metric, Recorded Value, Notes
Telemetry Metric	Recorded Value	Notes
MLflow Run ID	`0b4502a831aa4f499a00d38d23633896`	Experiment: `BioEvidence-LLM-SFT`
Total Tokens Processed	`2,420,268` (~2.42M tokens)	Complete token budget over 3 epochs
Mean Token Accuracy	`82.72%` (`0.8272`)	High next-token prediction fidelity

Step-by-Step Loss Convergence

Table with columns: Optimization Step, Training Loss, Learning Rate, Gradient Norm, Epoch Progress
Optimization Step	Training Loss	Learning Rate	Gradient Norm	Epoch Progress
`10`	1.6904	`2.00e-04`	0.64	0.09
`30`	0.8131	`1.97e-04`	0.19	0.27

📈 Visual Analytics & Convergence Plots

1. Training Loss Convergence & Cosine LR Schedule

Training Loss Curve

2. Comparative Evaluation Benchmark (Base vs Fine-Tuned)

Benchmark Comparison

3. Training Dataset Composition & Class Balance

Dataset Distribution

🏆 Quantitative Benchmark Evaluation

Evaluated on held-out test articles with zero PMID overlap with the training set:

Table with columns: Evaluation Metric, Base Model (Qwen2.5-1.5B-Instruct Zero-Shot), Fine-Tuned (BioEvidence-LLM-1.5B), Measured Delta / Gain, Clinical Impact
Evaluation Metric	Base Model (`Qwen2.5-1.5B-Instruct` Zero-Shot)	Fine-Tuned (`BioEvidence-LLM-1.5B`)	Measured Delta / Gain	Clinical Impact
Decision Accuracy	62.2%	78.2%	`+16.0%`	Accurately identifies `YES`, `NO`, or `MAYBE` findings.
Macro F1 Score

⚖️ Qualitative Output Comparison: Before vs. After Fine-Tuning

Sample 1: Rural vs. Urban Neonatal Mortality (PubMedQA: PMID 16428354)

Clinical Question: "Does rural or urban residence make a difference to neonatal outcome in premature birth?"
Source Evidence Excerpt: "Infants of rural residence had a higher mortality (adjusted odds ratio (OR) 1.26, 95% confidence interval (CI) 1.07 to 1.48, p = 0.005). Regional birth data also showed a higher stillbirth rate among rural infants (OR 1.20, 95% CI 1.09 to 1.32, p<0.001)."

Table with columns: Model Stage, Model Output Generation, Quality Assessment
Model Stage	Model Output Generation	Quality Assessment
⚠️ Before Fine-Tuning(Base Qwen2.5-1.5B)	`"Hello! Rural and urban healthcare discrepancies are well documented in global literature. Generally speaking, premature infants face complications like respiratory distress. Access to specialized hospitals can influence mortality. It is plausible that rural mothers face challenges. Consult public health guidelines for more information."`	❌ Deficiencies:• No JSON schema• No decision label (`YES`/`NO`/`MAYBE`)• Missing exact numbers (OR 1.26, $p = 0.005$ )• Conversational padding
✅ After Fine-Tuning(BioEvidence-LLM-1.5B)

Sample 2: Cervical Cancer Lymphadenectomy (PubMedQA: PMID 25859857)

Clinical Question: "Could the extent of lymphadenectomy be modified by neoadjuvant chemotherapy in cervical cancer?"
Source Evidence Excerpt: "We analyzed groups of 167 and 140 patients who were diagnosed with lymph node metastasis in the matched primary surgery group and NACT group, respectively, and no significant difference was observed (p = 0.081)."

Table with columns: Model Stage, Model Output Generation, Quality Assessment
Model Stage	Model Output Generation	Quality Assessment
⚠️ Before Fine-Tuning(Base Qwen2.5-1.5B)	`"Neoadjuvant chemotherapy (NACT) is widely used in oncology to downstage tumors. In cervical cancer, resection depends on patient response. While some studies suggest lymph node involvement decreases, extensive lymphadenectomy is usually still considered standard."`	❌ Deficiencies:• Vague general opinion• Missing explicit `NO` classification• Missed non-significant $p$ -value ( $p = 0.081$ )• No structured JSON
✅ After Fine-Tuning(BioEvidence-LLM-1.5B)	json\n{\n "decision": "no",\n "answer": "The frequency and topographic distribution of lymph node metastasis are not modified by neoadjuvant chemotherapy. Systematic and extensive lymphadenectomy remains necessary.",\n "evidence": [\n "We analyzed groups of 167 and 140 patients who were diagnosed with lymph node metastasis in the matched primary surgery group and NACT group, respectively, and no significant difference was observed (p = 0.081)."\n ],\n "uncertainty": "Clinical non-responders showed higher nodal involvement requiring uniform surgical margins.",\n "limitations": [\n "Retrospective matched-case study design"\n ]\n}\n

🍃 Environmental Impact & Compute Footprint

Hardware: NVIDIA GeForce RTX 3050 Laptop GPU (TGP ~60W)
Training Time: ~1.95 hours (7,045 seconds)
Estimated Energy Consumption: ~0.117 kWh
Estimated Carbon Emissions: ~0.045 kg CO₂ eq (based on regional grid average)

📖 Citation & Provenance

If you use BioEvidence-LLM in your biomedical NLP research, please cite:

bibtex

@misc{talhande2026bioevidencellm,  title={BioEvidence-LLM: Biomedical Evidence-Grounded Language Modeling with Preserved Uncertainty},  author={Talhande, Bhupati},  year={2026},  publisher={Hugging Face},  howpublished={\url{https://huggingface.co/Bhupati1998/BioEvidence-LLM-1.5B}}}

📬 Contact & Maintainer

Developer: Bhupati Talhande (@Bhupati1998)
Hugging Face Hub: https://huggingface.co/Bhupati1998/BioEvidence-LLM-1.5B
Datasets: https://huggingface.co/datasets/Bhupati1998/BioEvidence-Datasets

import jsonimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel BASE_MODEL_NAME = "Qwen/Qwen2.5-1.5B-Instruct"ADAPTER_REPO_ID = "Bhupati1998/BioEvidence-LLM-1.5B" # 1. Load Tokenizer & Base Modeltokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_NAME, trust_remote_code=True)base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL_NAME, torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32, device_map="auto" if torch.cuda.is_available() else None, trust_remote_code=True,) # 2. Attach Trained LoRA Adaptermodel = PeftModel.from_pretrained(base_model, ADAPTER_REPO_ID)model.eval() # 3. Define Biomedical Question & Evidence Contextquestion = "Does statin therapy reduce 30-day cardiovascular mortality in patients with type 2 diabetes?"context = ( "BACKGROUND: Cardiovascular events represent the primary source of excess mortality in diabetic patients. " "METHODS: In a multi-center randomized controlled trial of 1,200 diabetic adults, subjects were assigned to daily atorvastatin 20mg or placebo. " "RESULTS: At 30 days, cardiovascular mortality was 2.8% in the atorvastatin arm versus 5.1% in the placebo arm (hazard ratio 0.54, 95% CI 0.38-0.78, p=0.002). " "CONCLUSIONS: Statin therapy significantly reduces short-term cardiovascular mortality in diabetic adults.") # 4. Format Prompt with System Instructionmessages = [ { "role": "system", "content": ( "You are BioEvidence-LLM, a specialized biomedical evidence-grounded AI. " "Analyze the biomedical question strictly based on the provided evidence context. " "Output your answer ONLY as a valid JSON object with the keys: decision, answer, evidence, uncertainty, limitations." ), }, { "role": "user", "content": f"Evidence Context:\n{context}\n\nQuestion: {question}", },] prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) # 5. Generate Grounded JSON Responsewith torch.no_grad(): outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.1, do_sample=False) response_text = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)print(response_text)

{ "decision": "yes", "answer": "Statin therapy significantly reduces 30-day cardiovascular mortality in patients with type 2 diabetes.", "evidence": [ "At 30 days, cardiovascular mortality was 2.8% in the atorvastatin arm versus 5.1% in the placebo arm (hazard ratio 0.54, 95% CI 0.38-0.78, p=0.002)." ], "uncertainty": null, "limitations": [ "Trial evaluated short-term 30-day follow-up duration (n=1200)." ]}

Hyperparameter

Value

Description

Compute Hardware

NVIDIA GeForce RTX 3050 Laptop GPU

4,096 MiB GDDR6 VRAM

CPU Architecture

12th Gen Intel Core i7-12650H

10 Cores, 16 Threads

Quantization Scheme

4-bit NormalFloat4 (nf4)

Double Quantization enabled (bitsandbytes)

PEFT Method

LoRA (

r

, Dropout )

Telemetry Metric

Recorded Value

Notes

MLflow Run ID

0b4502a831aa4f499a00d38d23633896

Experiment: BioEvidence-LLM-SFT

Total Tokens Processed

2,420,268 (~2.42M tokens)

Complete token budget over 3 epochs

Mean Token Accuracy

82.72% (0.8272)

High next-token prediction fidelity

Optimization Step

Training Loss

Learning Rate

Gradient Norm

Epoch Progress

10

1.6904

2.00e-04

0.64

0.09

30

0.8131

1.97e-04

0.19

0.27

Evaluation Metric

Base Model (Qwen2.5-1.5B-Instruct Zero-Shot)

Fine-Tuned (BioEvidence-LLM-1.5B)

Measured Delta / Gain

Clinical Impact

Decision Accuracy

62.2%

78.2%

+16.0%

Accurately identifies YES, NO, or MAYBE findings.

Macro F1 Score

Model Stage

Model Output Generation

Quality Assessment

⚠️ Before Fine-Tuning(Base Qwen2.5-1.5B)

"Hello! Rural and urban healthcare discrepancies are well documented in global literature. Generally speaking, premature infants face complications like respiratory distress. Access to specialized hospitals can influence mortality. It is plausible that rural mothers face challenges. Consult public health guidelines for more information."

❌ Deficiencies:• No JSON schema• No decision label (YES/NO/MAYBE)• Missing exact numbers (OR 1.26,

p = 0.005

)• Conversational padding

✅ After Fine-Tuning(BioEvidence-LLM-1.5B)

Model Stage

Model Output Generation

Quality Assessment

⚠️ Before Fine-Tuning(Base Qwen2.5-1.5B)

"Neoadjuvant chemotherapy (NACT) is widely used in oncology to downstage tumors. In cervical cancer, resection depends on patient response. While some studies suggest lymph node involvement decreases, extensive lymphadenectomy is usually still considered standard."

❌ Deficiencies:• Vague general opinion• Missing explicit NO classification• Missed non-significant

p

-value (

p = 0.081

)• No structured JSON

✅ After Fine-Tuning(BioEvidence-LLM-1.5B)

json\n{\n  "decision": "no",\n  "answer": "The frequency and topographic distribution of lymph node metastasis are not modified by neoadjuvant chemotherapy. Systematic and extensive lymphadenectomy remains necessary.",\n  "evidence": [\n    "We analyzed groups of 167 and 140 patients who were diagnosed with lymph node metastasis in the matched primary surgery group and NACT group, respectively, and no significant difference was observed (p = 0.081)."\n  ],\n  "uncertainty": "Clinical non-responders showed higher nodal involvement requiring uniform surgical margins.",\n  "limitations": [\n    "Retrospective matched-case study design"\n  ]\n}\n

@misc{talhande2026bioevidencellm, title={BioEvidence-LLM: Biomedical Evidence-Grounded Language Modeling with Preserved Uncertainty}, author={Talhande, Bhupati}, year={2026}, publisher={Hugging Face}, howpublished={\url{https://huggingface.co/Bhupati1998/BioEvidence-LLM-1.5B}}}

BioEvidence-LLM-1.5B

README

⚠️ Mandatory Medical Safety Disclaimer

📋 Model Details

🎯 Intended Uses & Scope

Direct Use

Out-of-Scope Use

🚀 How to Get Started with the Model

📊 Output Schema Contract

⚙️ Training Details

Training Infrastructure & Hyperparameters

📈 Verified MLflow Run Telemetry (outputs/experiments/mlflow.db)

Step-by-Step Loss Convergence

📈 Visual Analytics & Convergence Plots

1. Training Loss Convergence & Cosine LR Schedule

2. Comparative Evaluation Benchmark (Base vs Fine-Tuned)

3. Training Dataset Composition & Class Balance

🏆 Quantitative Benchmark Evaluation

⚖️ Qualitative Output Comparison: Before vs. After Fine-Tuning

Sample 1: Rural vs. Urban Neonatal Mortality (PubMedQA: PMID 16428354)

Sample 2: Cervical Cancer Lymphadenectomy (PubMedQA: PMID 25859857)

🍃 Environmental Impact & Compute Footprint

📖 Citation & Provenance

📬 Contact & Maintainer

Explore FriendliAI today

README

⚠️ Mandatory Medical Safety Disclaimer

📋 Model Details

🎯 Intended Uses & Scope

Direct Use

Out-of-Scope Use

🚀 How to Get Started with the Model

📊 Output Schema Contract

⚙️ Training Details

Training Infrastructure & Hyperparameters

📈 Verified MLflow Run Telemetry (outputs/experiments/mlflow.db)

Step-by-Step Loss Convergence

📈 Visual Analytics & Convergence Plots

1. Training Loss Convergence & Cosine LR Schedule

2. Comparative Evaluation Benchmark (Base vs Fine-Tuned)

3. Training Dataset Composition & Class Balance

🏆 Quantitative Benchmark Evaluation

⚖️ Qualitative Output Comparison: Before vs. After Fine-Tuning

Sample 1: Rural vs. Urban Neonatal Mortality (PubMedQA: PMID 16428354)

Sample 2: Cervical Cancer Lymphadenectomy (PubMedQA: PMID 25859857)

🍃 Environmental Impact & Compute Footprint

📖 Citation & Provenance

📬 Contact & Maintainer

📈 Verified MLflow Run Telemetry (`outputs/experiments/mlflow.db`)

📈 Verified MLflow Run Telemetry (`outputs/experiments/mlflow.db`)