Model Description

• Base model: mistralai/Mistral-7B-Instruct-v0.2
• Fine-tuning method: QLoRA (4-bit NF4, r=16, alpha=32)
• Training data: 3,000 samples bootstrapped from CUAD via Gemini API
• Hardware: AWS SageMaker T4 GPU
• Adapter size: 54MB

Performance

Usage

python
Copy code
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-Instruct-v0.2",
    quantization_config=BitsAndBytesConfig(load_in_4bit=True),
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "Kamal1729/legal-clause-extractor-mistral")
tokenizer = AutoTokenizer.from_pretrained("Kamal1729/legal-clause-extractor-mistral")

def extract_clause(contract_text, clause_type):
    instruction = f"""You are a legal AI assistant. Analyze the following contract text 
and extract structured information about the {clause_type} clause present.

Contract Text:
{contract_text}

Return ONLY a valid JSON object with exactly these fields:
- clause_type, key_terms, obligations, risk_level, parties_involved"""

    prompt = f"<s>[INST] {instruction} [/INST]"
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
    outputs = model.generate(**inputs, max_new_tokens=256, do_sample=False)
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return response.split("[/INST]")[-1].strip()

Training Details

• Dataset: CUAD (500 contracts, 13K expert annotations, 41 clause categories)
• Labels bootstrapped using Gemini API
• Epochs: 3
• Learning rate: 2e-4
• Optimizer: paged_adamw_8bit
• LoRA rank: 16, alpha: 32
• Target modules: q_proj, k_proj, v_proj, o_proj

Supported Clause Types

41 categories including: Termination, Liability, IP Ownership, Non-Compete, Governing Law, Payment Terms, Audit Rights, Anti-Assignment, and more.