BrainHealthAI

BrainMed-8B-SFT

README

License: apache-2.0

Output format

The model is trained to reason inside <think>...</think> and answer inside <answer>...</answer>, under this system prompt:

text

You are a medical reasoning assistant. Work through the clinical problem step by step inside <think>...</think>, grounding every step in established medical knowledge, then give the final, complete answer inside <answer>...</answer>.

Evaluation (historical, descriptive)

Same harness for base and fine-tune: greedy decoding, strict answer prompt, historical max(head, tail) aggregate extraction with option-text and fuzzy fallbacks.

Table with columns: Benchmark, n (rows), HuatuoGPT-o1-8B (measured), BrainMed-8B-SFT, Difference (points)
Benchmark	n (rows)	HuatuoGPT-o1-8B (measured)	BrainMed-8B-SFT	Difference (points)
MedBullets op4	308	59.09	58.77	-0.32
MedBullets op5	308	55.19	56.82	+1.63
MedMCQA (val)	4183	63.30	61.20	-2.10
MedQA 4 options	1273	77.93	72.11	-5.82
MedXpertQA	1449	17.05	18.08	+1.03
PubMedQA	1000	80.20	77.10	-3.10
Macro6		58.79	57.35	-1.45

Rows are not independent cases (MedBullets op4/op5 share questions). Checkpoints were compared on these same tasks; no independent test set was held out. No confidence intervals are available because per-question predictions are not published here. The measured base is +2.89 points above its published values on average under this harness, so these numbers must not be ranked against scores transcribed from other papers.

A plausible, untested explanation for the regression: medical-o1 is part of the base model's own training pipeline, and this run applies further SFT after the base's RL stage. No controlled ablation has established the cause.

Full results, every archived run and limitations: BrainHealthAI/BrainMed-8B — results/REPORT.md.

Data integrity

Training rows: 44,351. Rows overlapping the evaluation benchmarks were kept (upstream setting). The overlap was measured and is declared below.

Table with columns: Benchmark, items, overlapping training rows
Benchmark	items	overlapping training rows
hle_med	103	30 (29.13%)
medbullets_op4	308	35 (11.36%)
medbullets_op5	308	35 (11.36%)
medqa_4opt	1273	1 (0.08%)
medqa_5opt	1273	1 (0.08%)
medxpertqa	1449

The "clean" score in the evaluation files excludes flagged items. It does not undo training exposure, unknown base-model exposure or selection on these tasks, and is not an unbiased estimate. This overlap originates upstream: the MedReason corpus derives reasoning traces from MMLU-medical, MedXpertQA and Humanity's Last Exam.

Answer-format alignment was applied to 17,083 rows (38.52%): a canonical The answer is X. was appended where the letter was unambiguously recoverable from options already present in the question. This is a formatting transformation, not physician validation of the traces.

Intended use

Research on medical-model adaptation and evaluation methodology only. Not for clinical use, decision support in care, or any decision about a real person.

Citation

bibtex

@software{brainmed8b2026,  title  = {BrainMed-8B: medical reasoning adaptation and evaluation pipeline},  author = {BrainResearch},  year   = {2026},  url    = {https://github.com/BrainHealthAI/BrainMed-8B}}@misc{wu2025medreason,  title={MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs},  author={Wu, Juncheng and others}, year={2025}, eprint={2504.00993}, archivePrefix={arXiv}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

BrainHealthAI

Model Tree

Base

FreedomIntelligence/HuatuoGPT-o1-8B

Fine-tuned

this model

Input Modalities

Text

Output Modalities