adityag-india

docbase-qwen3b-medical

README

License: apache-2.0

What this is

An experiment testing whether a small language model can be adapted for medical reasoning using only locally available hardware. The finding is that it can, measurably — and that the result falls well short of clinical usefulness.

Training

Table

Base model	`unsloth/Qwen2.5-3B-Instruct-bnb-4bit`
Method	QLoRA (4-bit base, LoRA adapters)
LoRA rank / alpha	16 / 32
Target modules	q, k, v, o, gate, up, down projections
Trainable params	~1% of 3B
Max sequence length	2,048
Epochs	3 (3,663 steps)
Wall clock	~13.3 h, interrupted and resumed
Batch size	2 × grad accum 8 (effective 16)
Learning rate	2e-4, linear, 20 warmup steps
Precision	bfloat16
Framework	Unsloth 2026.9.2, PyTorch 2.11.0+cu130
Seed	13

Hardware: NVIDIA RTX 4060 Laptop GPU (8GB), Intel Core i9-13900H, 16GB RAM, Windows.

Cost: approximately 13.3 hours wall-clock, 3.55 GB peak VRAM, USD 0.00.

The run halted at ~step 2,640 and was resumed from checkpoint-2600, losing 40 steps. metrics.json reports 231 minutes because it times a single process — that is the resumed segment (1,063 of 3,663 steps), not the run. The total is reconstructed from filesystem timestamps plus the measured 13.06 s/step of the resumed segment; see the note at the end of this card.

The peak VRAM figure is the interesting one — under half of an 8GB card. Domain adaptation of a 3B model does not require datacenter hardware. It does, on this hardware, require most of a day.

Data

FreedomIntelligence/medical-o1-reasoning-SFT, en config. 19,704 rows, 19,526 after filtering.

Each record has a question, an explicit chain-of-thought trace, and an answer. The reasoning trace being a supervision target is why this dataset was chosen — question-and-answer pairs alone cannot teach reasoning, since the intermediate inference is absent from the training signal.

Content caveat: this is Indian medical entrance examination material (NEET / AIIMS style), not clinical records or prescriptions. Performance here does not predict performance on patient documentation.

Evaluation

Eight held-out questions, never seen during training. Greedy decoding (do_sample=False) so runs are deterministic. Graded manually against reference answers — string matching was rejected because the model can reach a correct answer through fabricated reasoning, which automated scoring would mark correct.

Table with columns: #, Topic, Reference, Base, Tuned
#	Topic	Reference	Base	Tuned
1	HIV post-exposure prophylaxis	Option C	❌	❌
2	Facial nerve palsy	Bell's palsy	✅	✅
3	Chronic ear discharge	M. tuberculosis	❌	❌
4

Base: 3/8 (37.5%) → Fine-tuned: 5/8 (62.5%). Two gained, none lost.

Both runs used identical base weights, questions, prompt format and grader, so the improvement is attributable to the adapter alone.

The two gains differ in kind. On Q5 the base model confidently chose phenytoin; the tuned model identified vitamin A toxicity and reasoned to it correctly — genuine domain knowledge. On Q4 the base double-applied the duration term and returned 0.016; the tuned model applied prevalence = incidence × duration cleanly — corrected procedure.

Output format also shifted wholesale to the training data's first-person reasoning register. Consistent with an earlier run where 80 records changed format but not accuracy: format is cheap to teach, domain knowledge is expensive.

What did not improve

Q1 returned the identical wrong answer both times — the training data does not cover this material.
Q3 moved from one wrong answer to a different one (viral → fungal) without reaching tuberculous otitis media.
Q7 arguably regressed: the tuned model explicitly considered the correct answer and rejected it.
On Q4 the correct value was then restated as "8%", off by a factor of ten. Right answer, unreliable surrounding text.

Limitations

n=8. One question is worth 12.5%. The improvement is suggestive, not established.
Single-rater, unblinded grading. The grader knew which output came from which model.
Exam content, not clinical records.
Correct answers are not always correctly reasoned. Scoring final answers alone overstates capability.
No full fine-tuning comparison has been run, so the choice of LoRA is justified by reasoning rather than measurement.
Third epoch was wasted. Loss was 1.1836 at step 2,600 and 1.1941 at 3,663. The model converged by epoch 2; at 13.06 s/step an epoch is ~4.4 h, so two epochs would give an equivalent model in ~8.9 h instead of ~13.3.
Total training time is reconstructed, not measured. Nothing on disk records the wall clock before the halt; the figure assumes throughput was unchanged across the interruption. The hard upper bound on the pre-halt segment is 12.94 h.

Usage

python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    "adityag-india/docbase-qwen3b-medical",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

messages = [
    {"role": "system", "content": "You are a clinical reasoning assistant. Think through the case step by step, then give your final answer."},
    {"role": "user", "content": "A 30-year-old man presents with inability to close his left eye, tearing over the left cheek, and saliva dribbling from the left angle of his mouth. What is the most likely diagnosis?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Use the tokenizer's chat template rather than a hand-written prompt string. The model was trained on Qwen2.5's ChatML format; a mismatched prompt makes a working fine-tune look broken.

Reproducing

bash
python 04_train_unsloth.py --stage sft --config ../configs/8gb_3b.json

Seed 13 throughout, greedy decoding at evaluation.

Note on metrics.json: the run halted at ~step 2,640 and was restarted from checkpoint-2600, and this file was written by the resumed process. Both of its headline numbers therefore describe only that segment.

train_loss: 0.3467 is a Trainer artifact — loss is accumulated from the resume point but divided by total steps (1.1941 × 1063/3663 = 0.3465). The true final loss is 1.194.
minutes: 231.4 is the last 1,063 of 3,663 steps, 29% of training. The full run took roughly 13.3 hours.

Reconstructed from filesystem timestamps: run 1 started Sat 08:20, run 2 ran Sat 21:16 → Sun 01:08 (232 min by mtime, matching the 231.4 recorded — which is what confirms the file covers the resume only). The resumed segment measured 13.06 s per optimiser step.

If you reuse this pipeline, accumulate elapsed time across resumes rather than timing one process — read any existing metrics.json at startup and add its minutes to the new segment.

Citation

bibtex
@misc{docbase-qwen3b-medical,
  author = {Gollapalli, Aditya},
  title  = {docbase-qwen3b-medical: QLoRA adaptation of Qwen2.5-3B for medical reasoning},
  year   = {2026},
  note   = {LoRA adapter. Research artifact, not validated for clinical use.}
}

Base model: Qwen2.5-3B-Instruct (Alibaba, Apache 2.0). Training data: FreedomIntelligence/medical-o1-reasoning-SFT, from the HuatuoGPT-o1 project (arXiv:2412.18925).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

adityag-india

Model Tree

Base

unsloth/Qwen2.5-3B-Instruct-bnb-4bit

Adapter

this model

Input Modalities

Text

Output Modalities