Base model

Qwen/Qwen2.5-7B-Instruct

Training config

• QLoRA (4-bit NF4 + double quantization)
• LoRA r=64, α=128, RSLoRA scaling
• Target modules: q/k/v/o/gate/up/down projections
• 3 epochs, best checkpoint = epoch 2 (eval_loss 0.3665)
• ~56 hours on RTX PRO 6000 Blackwell 96GB

Results

Task A: Field-level Score (bootstrap 95% CI)

• IDT: 90.0% [87.7, 92.1]
• OOD: 65.0%

Task B: Statute Retrieval (Top-K Accuracy)

• IDT Top-1: 92.7%
• IDT Top-5: 95.6% [93.3, 97.7]
• OOD Top-5: 89.3% (deep generalization)

Usage

python
Copy code
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    quantization_config=bnb,
    device_map="auto",
)
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

model = PeftModel.from_pretrained(base, "Megan1234/judgment-ai-adapter")
model.eval()

Disclaimer

本研究為輔助分析,非法律建議。實際法律問題請諮詢律師。 使用司法院去識別化公開判決書資料,不可用於 re-identification。