Kamalesh-genai

phi3-mini-legal-clause-lora

Model Details

Base model: microsoft/Phi-3-mini-4k-instruct (3.8B params)
Method: QLoRA (4-bit NF4 quantization) + PEFT LoRA (r=16, alpha=32)
Task: Multi-class contract clause classification (8 categories: Parties, License Grant, Cap On Liability, Anti-Assignment, Audit Rights, Insurance, Expiration Date, Governing Law)
Training framework: Hugging Face transformers, peft, trl (SFTTrainer)
Hardware: Google Colab free tier, 1x T4 GPU (16GB VRAM)
Training data: ~1,000 clause examples from CUAD, 2 epochs
Adapter size: ~17.8 MB

Evaluation

98.00% accuracy on a 50-example held-out eval slice (exact-match classification)
Final training loss: 0.81, validation loss: 0.77, mean token accuracy: ~82.9%

How to Use

python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model_id = "microsoft/Phi-3-mini-4k-instruct"
adapter_id = "Kamalesh-genai/phi3-mini-legal-clause-lora"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id, quantization_config=bnb_config, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

prompt = (
    "<|user|>\nYou are a legal contract analyst. Classify the following "
    "contract clause into exactly ONE of these categories: Parties, "
    "License Grant, Cap On Liability, Anti-Assignment, Audit Rights, "
    "Insurance, Expiration Date, Governing Law.\n\n"
    "Clause:\n\"\"\"This Agreement shall be governed by the laws of the "
    "State of New York.\"\"\"\n\nCategory:<|end|>\n<|assistant|>\n"
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=10)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Training Details

Dataset: CUAD (Atticus Project), filtered to the 8 most frequent clause categories
LoRA config: r=16, alpha=32, dropout=0.05, target modules: q/k/v/o_proj, gate/up/down_proj
Optimizer: paged_adamw_8bit, lr=2e-4, cosine schedule
Batch size: 2 (effective 8 with gradient accumulation), 2 epochs

Limitations

Trained on a filtered subset (top-8 CUAD categories, ~1,000 examples), not the full 41-category CUAD benchmark
Classification only — does not extract clause spans (original CUAD QA task)
Evaluated with simple exact-match accuracy on a small held-out slice

Author

Fine-tuned by Kamaleshwar S as a portfolio project — training code on GitHub

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Kamalesh-genai

Model Tree

Base

microsoft/Phi-3-mini-4k-instruct

Adapter

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Model Details

Base model: microsoft/Phi-3-mini-4k-instruct (3.8B params)
Method: QLoRA (4-bit NF4 quantization) + PEFT LoRA (r=16, alpha=32)
Task: Multi-class contract clause classification (8 categories: Parties, License Grant, Cap On Liability, Anti-Assignment, Audit Rights, Insurance, Expiration Date, Governing Law)
Training framework: Hugging Face transformers, peft, trl (SFTTrainer)
Hardware: Google Colab free tier, 1x T4 GPU (16GB VRAM)
Training data: ~1,000 clause examples from CUAD, 2 epochs
Adapter size: ~17.8 MB

Evaluation

98.00% accuracy on a 50-example held-out eval slice (exact-match classification)
Final training loss: 0.81, validation loss: 0.77, mean token accuracy: ~82.9%

How to Use

python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model_id = "microsoft/Phi-3-mini-4k-instruct"
adapter_id = "Kamalesh-genai/phi3-mini-legal-clause-lora"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id, quantization_config=bnb_config, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

prompt = (
    "<|user|>\nYou are a legal contract analyst. Classify the following "
    "contract clause into exactly ONE of these categories: Parties, "
    "License Grant, Cap On Liability, Anti-Assignment, Audit Rights, "
    "Insurance, Expiration Date, Governing Law.\n\n"
    "Clause:\n\"\"\"This Agreement shall be governed by the laws of the "
    "State of New York.\"\"\"\n\nCategory:<|end|>\n<|assistant|>\n"
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=10)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Training Details

Dataset: CUAD (Atticus Project), filtered to the 8 most frequent clause categories
LoRA config: r=16, alpha=32, dropout=0.05, target modules: q/k/v/o_proj, gate/up/down_proj
Optimizer: paged_adamw_8bit, lr=2e-4, cosine schedule
Batch size: 2 (effective 8 with gradient accumulation), 2 epochs

Limitations

Trained on a filtered subset (top-8 CUAD categories, ~1,000 examples), not the full 41-category CUAD benchmark
Classification only — does not extract clause spans (original CUAD QA task)
Evaluated with simple exact-match accuracy on a small held-out slice

Author

Fine-tuned by Kamaleshwar S as a portfolio project — training code on GitHub

phi3-mini-legal-clause-lora

README

Model Details

Evaluation

How to Use

Training Details

Limitations

Author

Explore FriendliAI today

README

Model Details

Evaluation

How to Use

Training Details

Limitations

Author