musk1209

finsight-qlora-mistral

README

License: apache-2.0

Performance

Table with columns: metric, value
metric	value
Train loss (final epoch)	0.8116
Eval loss (held-out val set)	0.8674
Eval perplexity	2.38
Training time	330.0 minutes
Training samples/sec	1.01

Training Setup

Table with columns: Setting, Value
Setting	Value
Base model	`mistralai/Mistral-7B-Instruct-v0.3`
Dataset	`virattt/financial-qa-10K`
Training records	6,647
Validation records	350
Method	QLoRA (4-bit quantized base + LoRA adapters)
Quantization	NF4 double-quant, fp16 compute
LoRA rank (r)	16
LoRA alpha	32
LoRA target modules	q_proj, k_proj, v_proj, o_proj
Epochs	3
Effective batch size	16
Learning rate	0.0002
Optimizer	AdamW (torch)
LR schedule	Cosine, 3% warmup
Seed	42
Hardware	NVIDIA Tesla T4 (Kaggle)

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

BASE = "mistralai/Mistral-7B-Instruct-v0.3"

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)
tokenizer = AutoTokenizer.from_pretrained(BASE)
base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, "musk1209/finsight-qlora-mistral")

context = "The Company designs, manufactures and markets smartphones..."
question = "What does the company sell?"
prompt = f"[INST] Answer the question based on the provided context from a company's 10-K filing. Be concise and factual.\n\nContext: {context}\n\nQuestion: {question} [/INST]"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Observations

Head-to-head inference tests show that the fine-tuned model produces slightly more disclosure-style, complete answers than the base Mistral 7B, but the base model already performs well on this extractive QA task. This is consistent with modern instruction-tuned LLMs being strong on extractive QA out-of-the-box; fine-tuning provides refinement rather than enabling entirely new capabilities.

Limitations

Dataset origin: Training data is derived from 2023 10-K filings. Performance on other filing types (10-Q, 8-K, DEF 14A) or on filings outside 2023 may differ.
Extractive bias: The model is trained to answer questions where the answer is contained in the provided context. Open-ended synthesis across multiple documents is not part of its training.
Compute-time trade-off: Producing this adapter required ~6 hours of T4 GPU time. In practice, for many production use cases, the base Mistral 7B Instruct v0.3 with careful prompting delivers comparable quality without fine-tuning.

Citation

@misc{finsight-qlora-mistral, title = {FinSight: QLoRA-tuned Mistral 7B for Financial Question Answering}, author = {Tongaria, Muskan}, year = {2026}, url = {https://huggingface.co/musk1209/finsight-qlora-mistral} }

Honest evaluation vs base model (Path A vs Path B)

The finetuned adapter was compared head-to-head against the base Mistral 7B Instruct v0.3 with a strong system prompt (no adapter), on 6 test cases spanning basic extraction, numerical questions, risk factors, adversarial "answer not in context" prompts, exact-figure questions, and multi-fact synthesis.

Result: Path B (prompted base model) was chosen for deployment. Both approaches performed comparably on standard extraction, but the adapter demonstrated two critical failure modes:

Hallucination on unanswerable questions. When asked about CEO compensation given only a context about products, the adapter invented a specific figure ("$39,629"). The prompted base model correctly responded that the context did not contain the information.
Loss of reasoning on synthesis. When asked which of two segments performed better given growth rates for each, the adapter answered "Segment A" with no justification. The prompted base model gave the reasoning ("Segment A performed better, as it experienced a 12% increase compared to Segment B's 3% decrease").

Why this happened: the training dataset contains only answerable QA pairs where the answer is present in the context. The model was never trained on "should refuse" examples, so it learned to always produce an answer. This is a data-composition issue, not a QLoRA issue.

The full comparison results are in the project repository: data/path_comparison_results.json.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

musk1209

Model Tree

Base

mistralai/Mistral-7B-Instruct-v0.3

Adapter

this model

Input Modalities

Text

Output Modalities