BrainHealthAI

BrainMed-8B-SFT

README

License: apache-2.0

Output format

The model is trained to reason inside <think>...</think> and answer inside <answer>...</answer>, under this system prompt:

markdown
You are a medical reasoning assistant. Work through the clinical problem step by step inside <think>...</think>, grounding every step in established medical knowledge, then give the final, complete answer inside <answer>...</answer>.

Evaluation

Table with columns: Benchmark, n, Accuracy
Benchmark	n	Accuracy
medbullets_op4	308	58.77
medbullets_op5	308	56.82
medmcqa_val	4183	61.2
medqa_4opt	1273	72.11
medxpertqa	1449	18.08
pubmedqa_test	1000	77.1

Scored with the MedReason evaluation harness (greedy decoding, strict answer prompt, max(head, tail) extraction) on the benchmark files shipped by the MedReason and HuatuoGPT-o1 repositories - no benchmark was rebuilt or resampled.

Data integrity

Training rows: 44,351.

Rows overlapping the evaluation benchmarks were kept, matching the upstream setting under which the published comparison tables were produced. The overlap was measured and is declared below rather than removed; the evaluation therefore reports a clean-subset score alongside the raw one, and the clean figure is the unbiased estimate.

Table with columns: Benchmark, items, overlapping training rows
Benchmark	items	overlapping training rows
hle_med	103	30 (29.13%)
medbullets_op4	308	35 (11.36%)
medbullets_op5	308	35 (11.36%)
medqa_4opt	1273	1 (0.08%)
medqa_5opt	1273	1 (0.08%)
medxpertqa	1449

This overlap originates upstream: the MedReason corpus derives reasoning traces from MMLU-medical, MedXpertQA and Humanity's Last Exam, three sets it also evaluates on.

Answer-format alignment was applied to 17,083 rows (38.52%): a canonical The answer is X. was appended where the letter was unambiguously recoverable from options already present in the question. Questions and reasoning traces are byte-identical to the source.

Intended use

Research and decision support. Not a medical device; not for autonomous clinical use.

Citation

bibtex
@misc{wu2025medreason,
  title={MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs},
  author={Wu, Juncheng and others}, year={2025}, eprint={2504.00993}, archivePrefix={arXiv}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

BrainHealthAI

Model Tree

Base

FreedomIntelligence/HuatuoGPT-o1-8B

Fine-tuned

this model

Input Modalities

Text

Output Modalities