Rumiii

LlamaTron-RS1-Nemesis-1B

README

License: apache-2.0

Model Overview

LlamaTron RS1 Nemesis is a medical reasoning model produced by fine-tuning meta-llama/Llama-3.2-1B-Instruct on the Medical-Reasoning-SFT-MiniMax-M2.1 dataset using QLoRA. The dataset contains 204,773 clinical reasoning conversations with full chain-of-thought traces covering differential diagnosis, treatment planning, pharmacology, and clinical case analysis.

Despite being a 1 billion parameter model, it handles complex clinical questions with structured and coherent reasoning.

Training Results

Table with columns: Step, Train Loss, Validation Loss
Step	Train Loss	Validation Loss
500	1.5759	1.6126
1000	1.5176	1.5538
1500	1.4805	1.5256
2000	1.4795	1.5060
2500	1.4508	1.4939
3000	1.4534	1.4815
3500	1.4384	1.4739
4000	1.4228	1.4663
4500	1.4251	1.4605
5000	1.4301	1.4567
5500	1.4102	1.4545
6000	1.4246	1.4538
6271	1.4200	1.4500

Loss decreased consistently across all steps with train and validation loss tracking closely. No overfitting observed.

Dataset

Trained on Medical-Reasoning-SFT-MiniMax-M2.1 released by Maziyar Panahi under the OpenMed initiative.

Table with columns: Property, Value
Property	Value
Total Samples	204,773
Estimated Tokens	~621 Million
Format	Multi-turn chat with chain-of-thought reasoning
License	Apache 2.0
Topics	Differential diagnosis, treatment planning, pharmacology, clinical case analysis

How to Use

Load the Model

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

model_id = "Rumiii/LlamaTron_RS1_Nemesis_1B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)

messages = [
    {
        "role": "system",
        "content": "You are LlamaTron RS1 Nemesis, a knowledgeable and compassionate medical AI assistant. Provide accurate, evidence-based medical information clearly and helpfully."
    },
    {
        "role": "user",
        "content": "What are the early symptoms of Type 2 Diabetes?"
    },
]

output = pipe(
    messages,
    max_new_tokens=400,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
)

print(output[0]["generated_text"][-1]["content"])

Repository

The full training code, merging scripts, and inference interface are available on GitHub: github.com/sufirumii/LlamaTron-RS1-Nemesis-1B

Limitations

This model is intended for research and educational purposes only
It is not a substitute for professional medical advice, diagnosis, or treatment
The model was trained with a maximum sequence length of 512 tokens which may limit performance on longer clinical texts
Always consult a qualified healthcare provider for medical decisions

Credits

Dataset: Maziyar Panahi and the OpenMed initiative for releasing the Medical-Reasoning-SFT-MiniMax-M2.1 dataset under Apache 2.0
Base Model: Meta AI for releasing Llama-3.2-1B-Instruct
Libraries: Hugging Face Transformers, PEFT, TRL, BitsAndBytes, Accelerate

License

Apache 2.0 — see LICENSE for details.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Rumiii

Model Tree

Base

meta-llama/Llama-3.2-1B-Instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities