Varun95

medical-qwen-lora

README

License: apache-2.0

Overview

MedReason is a LoRA adapter for Qwen2.5-3B-Instruct fine-tuned on medical-o1-reasoning-SFT. Given a patient vignette, it writes its reasoning inside <think> tags and then commits to a concise clinical answer. It was trained end-to-end on a single free-tier T4 GPU.

Table

Base model	Qwen2.5-3B-Instruct, 4-bit (Unsloth bnb)
Method	QLoRA, rank 16, alpha 16, dropout 0.05
Task	Single-turn clinical reasoning, English
Output	`<think>` reasoning block, then final answer
Status	Under-trained: 0.53 epoch of a planned 3

Quick start

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel BASE = "unsloth/Qwen2.5-3B-Instruct-bnb-4bit"base  = AutoModelForCausalLM.from_pretrained(BASE, device_map="auto", torch_dtype=torch.float16)tok   = AutoTokenizer.from_pretrained(BASE)model = PeftModel.from_pretrained(base, "@@REPO@") SYSTEM = ("You are a Clinical Decision Support AI. Analyze the patient information "          "methodically before providing a structured clinical summary.")messages = [    {"role": "system", "content": SYSTEM},    {"role": "user",   "content": "A 45-year-old male presents with sudden chest pain radiating "                                  "to the left arm, diaphoresis and shortness of breath. "                                  "History of hypertension and smoking."},]ids = tok.apply_chat_template(messages, tokenize=True, add_generation_prompt=True,                              return_tensors="pt").to(model.device)out = model.generate(input_ids=ids, max_new_tokens=1500, temperature=0.7, top_p=0.9, do_sample=True) text = tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)reasoning, _, answer = text.partition("</think>")print(answer.strip())

python

from unsloth import FastLanguageModelmodel, tok = FastLanguageModel.from_pretrained("Varun95/medical-qwen-lora", max_seq_length=4096, load_in_4bit=True)FastLanguageModel.for_inference(model)

Use the system prompt above. The adapter was trained with it, and behaviour without it is untested.

Training

Validation loss and the learning-rate schedule across both GPU sessions:

Table with columns: Step, Validation loss
Step	Validation loss
100	1.4988
500	1.4324
800	1.4171
1,000	1.4116
1,038	1.4116

Validation loss was still falling when the free GPU session budget ended. Training was stopped for time, not convergence.

Data

19,526 examples remained after cleaning (null, short-trace and duplicate removal; nothing exceeded 4,096 tokens). Split 80/10/10 with a fixed seed: 15,620 train / 1,952 validation / 1,954 held-out test. The test split has not been used. A pair-level check confirmed zero overlap between splits.

Configuration

Table with columns: Parameter, Value
Parameter	Value
LoRA targets	`q_proj k_proj v_proj o_proj gate_proj up_proj down_proj`
Learning rate	2e-4, cosine schedule, 3% warmup
Effective batch size	8 (2 per device x 4 accumulation)
Optimiser	`adamw_8bit`, weight decay 0.01
Precision	fp16, gradient checkpointing
Max sequence length	4,096
Loss	Assistant tokens only (prompt masked)
Frameworks

Adapter architecture

Only the low-rank branches in the diagram below receive gradients. Every attention and MLP projection — q_proj k_proj v_proj o_proj gate_proj up_proj down_proj — in all 36 transformer blocks carries a pair of rank-16 matrices A and B, while the 4-bit base weights W₀ stay frozen. The branch output is scaled by α/r = 1, so each adapter acts as a clean additive residual on top of its projection. That totals 29.9M trainable parameters, roughly 0.97% of the 3.09B-parameter backbone, which is what lets the whole run fit on a single free-tier T4.

Qualitative check

Three vignettes were sampled at step 1,038 (temperature 0.7): chest pain (45M), paediatric fever (3Y), and right-lower-quadrant pain (28F). All three produced a closed <think> block followed by a final answer. This verifies output format only and says nothing about clinical accuracy. No benchmark evaluation has been run yet.

Limitations

Not clinically validated. Fluent reasoning can still be wrong or unsafe. Check outputs against NICE, WHO or local guidelines.
Synthetic supervision. The reasoning traces in the training data were machine-generated, so the model can reproduce conversational filler that does not fit a single vignette.
Under-trained. Roughly half an epoch was completed.
Hallucination. Invented facts and over-confident differentials are possible.
Scope. English, single-turn, short vignettes only.

Citation

bibtex

@misc{mulay2026medreason,  author    = {Mulay, Varun},  title     = {MedReason: A Clinical Reasoning LoRA for Qwen2.5-3B},  year      = {2026},  publisher = {Hugging Face},  url       = {https://huggingface.co/Varun95/medical-qwen-lora}}

Fine-tuned on a free Colab T4 with Unsloth. Base model and dataset licences apply.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Varun95

Model Tree

Base

unsloth/Qwen2.5-3B-Instruct-bnb-4bit

Adapter

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Overview

Table

Base model	Qwen2.5-3B-Instruct, 4-bit (Unsloth bnb)
Method	QLoRA, rank 16, alpha 16, dropout 0.05
Task	Single-turn clinical reasoning, English
Output	`<think>` reasoning block, then final answer
Status	Under-trained: 0.53 epoch of a planned 3

Quick start

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel BASE = "unsloth/Qwen2.5-3B-Instruct-bnb-4bit"base  = AutoModelForCausalLM.from_pretrained(BASE, device_map="auto", torch_dtype=torch.float16)tok   = AutoTokenizer.from_pretrained(BASE)model = PeftModel.from_pretrained(base, "@@REPO@") SYSTEM = ("You are a Clinical Decision Support AI. Analyze the patient information "          "methodically before providing a structured clinical summary.")messages = [    {"role": "system", "content": SYSTEM},    {"role": "user",   "content": "A 45-year-old male presents with sudden chest pain radiating "                                  "to the left arm, diaphoresis and shortness of breath. "                                  "History of hypertension and smoking."},]ids = tok.apply_chat_template(messages, tokenize=True, add_generation_prompt=True,                              return_tensors="pt").to(model.device)out = model.generate(input_ids=ids, max_new_tokens=1500, temperature=0.7, top_p=0.9, do_sample=True) text = tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)reasoning, _, answer = text.partition("</think>")print(answer.strip())

python

from unsloth import FastLanguageModelmodel, tok = FastLanguageModel.from_pretrained("Varun95/medical-qwen-lora", max_seq_length=4096, load_in_4bit=True)FastLanguageModel.for_inference(model)

Use the system prompt above. The adapter was trained with it, and behaviour without it is untested.

Training

Validation loss and the learning-rate schedule across both GPU sessions:

Table with columns: Step, Validation loss
Step	Validation loss
100	1.4988
500	1.4324
800	1.4171
1,000	1.4116
1,038	1.4116

Validation loss was still falling when the free GPU session budget ended. Training was stopped for time, not convergence.

Data

Configuration

Table with columns: Parameter, Value
Parameter	Value
LoRA targets	`q_proj k_proj v_proj o_proj gate_proj up_proj down_proj`
Learning rate	2e-4, cosine schedule, 3% warmup
Effective batch size	8 (2 per device x 4 accumulation)
Optimiser	`adamw_8bit`, weight decay 0.01
Precision	fp16, gradient checkpointing
Max sequence length	4,096
Loss	Assistant tokens only (prompt masked)
Frameworks

Adapter architecture

Qualitative check

Limitations

Not clinically validated. Fluent reasoning can still be wrong or unsafe. Check outputs against NICE, WHO or local guidelines.
Synthetic supervision. The reasoning traces in the training data were machine-generated, so the model can reproduce conversational filler that does not fit a single vignette.
Under-trained. Roughly half an epoch was completed.
Hallucination. Invented facts and over-confident differentials are possible.
Scope. English, single-turn, short vignettes only.

Citation

bibtex

@misc{mulay2026medreason,  author    = {Mulay, Varun},  title     = {MedReason: A Clinical Reasoning LoRA for Qwen2.5-3B},  year      = {2026},  publisher = {Hugging Face},  url       = {https://huggingface.co/Varun95/medical-qwen-lora}}

Fine-tuned on a free Colab T4 with Unsloth. Base model and dataset licences apply.