rohan1324

phi3-mini-finance-qlora

Deploy Dedicated

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

README

License: mit

Training Details

Base model: microsoft/Phi-3-mini-4k-instruct (3.8B parameters)
Method: QLoRA (LoRA rank=16, 4-bit NF4 quantization)
Dataset: gbharti/finance-alpaca (5,000 samples)
Training steps: 500
Hardware: NVIDIA T4 16GB (Kaggle free tier)
Training time: ~45 minutes

Results

Table
Model	ROUGE-1	ROUGE-L	Latency	Cost/query
Base Phi-3 Mini	0.238	0.131	11.4s	$0.000
Fine-tuned Phi-3 Mini	0.223	0.146	10.5s	$0.000
GPT-4o (Azure)	0.282	0.133	3.1s	~$0.030

+11.5% ROUGE-L improvement over base model at zero inference cost.

Usage

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

tokenizer = AutoTokenizer.from_pretrained("rohan1324/phi3-mini-finance-qlora")

base = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=False,
    attn_implementation="eager"
)

model = PeftModel.from_pretrained(base, "rohan1324/phi3-mini-finance-qlora")

prompt = "### Instruction:\nWhat is EBITDA?\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))