afzal2003

finllm-sentiment-qlora

README

License: apache-2.0

Results

Evaluated on a held-out test set of 227 examples from FinancialPhraseBank (all-agreement subset), using the identical prompt template and zero-shot for all frontier models (i.e. they were not fine-tuned — that asymmetry is intentional, see Limitations).

Table with columns: Model, Macro F1, Avg Latency (ms), P95 Latency (ms), Cost / 1,000 classifications
Model	Macro F1	Avg Latency (ms)	P95 Latency (ms)	Cost / 1,000 classifications
This model (QLoRA)	0.9722	349.3	392.3	$0.00*
Claude Haiku 4.5	0.9252	800.1	1,003.8	$0.2704
GPT-6 Luna	0.9248	1,031.1	1,477.0	$0.068

* Local inference cost is amortized GPU/electricity cost, not a per-call API charge — not directly comparable to the hosted-API columns, but effectively negligible at this scale on consumer hardware (RTX 4060, 8GB VRAM).

Takeaway: the fine-tuned model matched or exceeded both frontier APIs on accuracy while being 2.3–3x faster per call and free to run at inference time, at the cost of being narrowly specialized to this one task.

Gemini was excluded from this comparison due to persistent Google Cloud billing-verification issues encountered during testing (see Limitations).

Model Details

Base model: Qwen/Qwen2.5-7B-Instruct
Fine-tuning method: QLoRA (4-bit NF4 quantization, double quantization, LoRA adapters on attention projections)
LoRA config: r=16, alpha=32, dropout=0.05, target modules: q_proj, k_proj, v_proj, o_proj
Trainable parameters: 10,092,544 / 7,625,709,056 total (0.13%)
Training hardware: single NVIDIA RTX 4060 Laptop GPU (8GB VRAM), via WSL2
Training time: ~2.5 hours for 3 epochs
Task: 3-class sentiment classification (negative / neutral / positive)

Training Data

FinancialPhraseBank, all-agreement subset (sentences where all annotators agreed on the label) — 2,264 examples total.

Train: 1,811 examples
Validation: 226 examples
Test: 227 examples (held out, never seen during training)

Split with a fixed random seed for reproducibility.

Hardware Requirements

Training and quantized local inference require an NVIDIA GPU with CUDA support. This project was built and tested end-to-end on a single RTX 4060 Laptop GPU (8GB VRAM) under WSL2. The 4-bit quantization path relies on bitsandbytes, which is CUDA-specific.

macOS / Apple Silicon is not currently supported for training or for the quantized inference path shown below — bitsandbytes has no functional Apple Silicon (Metal/MPS) support, so BitsAndBytesConfig will fail on a Mac.
A Mac (or any CUDA-less machine) can still run this adapter, but only by loading the base model in full fp16/bf16 precision instead of 4-bit — skipping BitsAndBytesConfig entirely and using device_map="mps" (Apple Silicon) or device_map="cpu". This needs considerably more memory (Qwen2.5-7B in fp16 needs roughly 15GB+ of unified memory/RAM, ideally 24GB+ for comfortable headroom) and runs slower without quantization.
The frontier-API comparison code (the GPT and Claude portions of the evaluation harness) has no GPU dependency at all and runs on any platform, since it's just network API calls.
Minimum VRAM for the 4-bit path (NVIDIA only): 8GB was sufficient for this project's settings (short sequence length, batch size 1, gradient checkpointing); less than 8GB will likely require reducing max_seq_length or LoRA rank further.

Training Procedure

Quantization: 4-bit NF4, double quantization, bfloat16 compute dtype
Batch size: 1, with gradient accumulation over 8 steps (effective batch size 8)
Epochs: 3
Learning rate: 2e-4, cosine schedule, 3% warmup
Optimizer: paged AdamW (8-bit) — needed to fit training within 8GB VRAM
Gradient checkpointing: enabled

Training loss dropped from 3.32 to approximately 1.0 over the course of training; validation loss tracked closely with training loss throughout, with no signs of overfitting.

How to Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModelimport torch base_model_name = "Qwen/Qwen2.5-7B-Instruct"adapter_name = "afzal2003/finllm-sentiment-qlora" bnb_config = BitsAndBytesConfig(    load_in_4bit=True,    bnb_4bit_quant_type="nf4",    bnb_4bit_use_double_quant=True,    bnb_4bit_compute_dtype=torch.bfloat16,) tokenizer = AutoTokenizer.from_pretrained(base_model_name)base_model = AutoModelForCausalLM.from_pretrained(    base_model_name, quantization_config=bnb_config, device_map="auto")model = PeftModel.from_pretrained(base_model, adapter_name)model.eval() prompt = (    "Classify the sentiment of the following financial statement as "    "exactly one of: negative, neutral, positive.\n\n"    "Statement: The company reported record profits this quarter.\n\n"    "Sentiment:") inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=5, do_sample=False)print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

(The snippet above requires an NVIDIA GPU per the Hardware Requirements section. On a Mac or CPU-only machine, drop quantization_config=bnb_config and use device_map="mps" or device_map="cpu" instead.)

Evaluation Methodology

All models were evaluated on the same 227-example test set, using the identical prompt template shown above, with greedy/deterministic decoding where the API allowed it. Frontier models were called zero-shot via their respective APIs (OpenAI, Anthropic) — no fine-tuning or few-shot examples were provided to them, so this comparison measures this model's fine-tuned specialization against general-purpose models used out of the box, not two fine-tuned systems against each other.

Models compared: GPT-6 Luna (gpt-6-luna), Claude Haiku 4.5 (claude-haiku-4-5-20251001)
Metric: Macro F1 (unweighted average across the three classes, to avoid the majority "neutral" class dominating the score)
Latency: wall-clock time per API/inference call, including network round-trip for hosted models
Cost: computed from each provider's published per-token pricing at the time of testing (September 2026); API pricing changes over time, so these figures should be treated as a snapshot, not a permanent benchmark
Evaluated: September 2026

Limitations

Narrow task scope. This adapter is specialized for short-form financial sentiment classification and is not a general-purpose instruction-following model. It should not be expected to perform well outside this task.
Small test set. 227 examples is sufficient to demonstrate the comparison methodology but is not large enough to make strong statistical claims about the true performance gap between models.
Frontier models were not fine-tuned. This is an intentional design choice (the fine-tuned model has an inherent advantage on its specific training distribution), not an oversight — but it means this is not a fair comparison of "best possible GPT/Claude performance," only "GPT/Claude used as a general-purpose classifier out of the box."
Gemini was not included. Testing was blocked by a Google Cloud billing-account verification failure unrelated to the model or code itself; this is not a statement about Gemini's actual performance on this task.
Pricing is a snapshot. Frontier API model names and pricing changed multiple times over the course of this project (weeks), and will likely have changed again by the time you're reading this — verify current pricing before drawing cost conclusions.
Hardware. See Hardware Requirements above — training and 4-bit inference require an NVIDIA GPU; macOS/Apple Silicon users can only run the unquantized fp16/bf16 path.

Reproducibility

Full training and evaluation code (data preparation, QLoRA training script, evaluation harness) is available on GitHub: afzall003/finllm-sentiment-qlora.

Citation

If you use this adapter, please cite the base model and the FinancialPhraseBank dataset:

text

@article{malo2014good,  title={Good debt or bad debt: Detecting semantic orientations in economic texts},  author={Malo, Pekka and Sinha, Ankur and Korhonen, Pekka and Wallenius, Jyrki and Takala, Pyry},  journal={Journal of the Association for Information Science and Technology},  year={2014}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

afzal2003

Model Tree

Base

Qwen/Qwen2.5-7B-Instruct

Adapter

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Results

Table with columns: Model, Macro F1, Avg Latency (ms), P95 Latency (ms), Cost / 1,000 classifications
Model	Macro F1	Avg Latency (ms)	P95 Latency (ms)	Cost / 1,000 classifications
This model (QLoRA)	0.9722	349.3	392.3	$0.00*
Claude Haiku 4.5	0.9252	800.1	1,003.8	$0.2704
GPT-6 Luna	0.9248	1,031.1	1,477.0	$0.068

Gemini was excluded from this comparison due to persistent Google Cloud billing-verification issues encountered during testing (see Limitations).

Model Details

Base model: Qwen/Qwen2.5-7B-Instruct
Fine-tuning method: QLoRA (4-bit NF4 quantization, double quantization, LoRA adapters on attention projections)
LoRA config: r=16, alpha=32, dropout=0.05, target modules: q_proj, k_proj, v_proj, o_proj
Trainable parameters: 10,092,544 / 7,625,709,056 total (0.13%)
Training hardware: single NVIDIA RTX 4060 Laptop GPU (8GB VRAM), via WSL2
Training time: ~2.5 hours for 3 epochs
Task: 3-class sentiment classification (negative / neutral / positive)

Training Data

FinancialPhraseBank, all-agreement subset (sentences where all annotators agreed on the label) — 2,264 examples total.

Train: 1,811 examples
Validation: 226 examples
Test: 227 examples (held out, never seen during training)

Split with a fixed random seed for reproducibility.

Hardware Requirements

macOS / Apple Silicon is not currently supported for training or for the quantized inference path shown below — bitsandbytes has no functional Apple Silicon (Metal/MPS) support, so BitsAndBytesConfig will fail on a Mac.
A Mac (or any CUDA-less machine) can still run this adapter, but only by loading the base model in full fp16/bf16 precision instead of 4-bit — skipping BitsAndBytesConfig entirely and using device_map="mps" (Apple Silicon) or device_map="cpu". This needs considerably more memory (Qwen2.5-7B in fp16 needs roughly 15GB+ of unified memory/RAM, ideally 24GB+ for comfortable headroom) and runs slower without quantization.
The frontier-API comparison code (the GPT and Claude portions of the evaluation harness) has no GPU dependency at all and runs on any platform, since it's just network API calls.
Minimum VRAM for the 4-bit path (NVIDIA only): 8GB was sufficient for this project's settings (short sequence length, batch size 1, gradient checkpointing); less than 8GB will likely require reducing max_seq_length or LoRA rank further.

Training Procedure

Quantization: 4-bit NF4, double quantization, bfloat16 compute dtype
Batch size: 1, with gradient accumulation over 8 steps (effective batch size 8)
Epochs: 3
Learning rate: 2e-4, cosine schedule, 3% warmup
Optimizer: paged AdamW (8-bit) — needed to fit training within 8GB VRAM
Gradient checkpointing: enabled

Training loss dropped from 3.32 to approximately 1.0 over the course of training; validation loss tracked closely with training loss throughout, with no signs of overfitting.

How to Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModelimport torch base_model_name = "Qwen/Qwen2.5-7B-Instruct"adapter_name = "afzal2003/finllm-sentiment-qlora" bnb_config = BitsAndBytesConfig(    load_in_4bit=True,    bnb_4bit_quant_type="nf4",    bnb_4bit_use_double_quant=True,    bnb_4bit_compute_dtype=torch.bfloat16,) tokenizer = AutoTokenizer.from_pretrained(base_model_name)base_model = AutoModelForCausalLM.from_pretrained(    base_model_name, quantization_config=bnb_config, device_map="auto")model = PeftModel.from_pretrained(base_model, adapter_name)model.eval() prompt = (    "Classify the sentiment of the following financial statement as "    "exactly one of: negative, neutral, positive.\n\n"    "Statement: The company reported record profits this quarter.\n\n"    "Sentiment:") inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=5, do_sample=False)print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModelimport torch base_model_name = "Qwen/Qwen2.5-7B-Instruct"adapter_name = "afzal2003/finllm-sentiment-qlora" bnb_config = BitsAndBytesConfig(    load_in_4bit=True,    bnb_4bit_quant_type="nf4",    bnb_4bit_use_double_quant=True,    bnb_4bit_compute_dtype=torch.bfloat16,) tokenizer = AutoTokenizer.from_pretrained(base_model_name)base_model = AutoModelForCausalLM.from_pretrained(    base_model_name, quantization_config=bnb_config, device_map="auto")model = PeftModel.from_pretrained(base_model, adapter_name)model.eval() prompt = (    "Classify the sentiment of the following financial statement as "    "exactly one of: negative, neutral, positive.\n\n"    "Statement: The company reported record profits this quarter.\n\n"    "Sentiment:") inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=5, do_sample=False)print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Evaluation Methodology

Models compared: GPT-6 Luna (gpt-6-luna), Claude Haiku 4.5 (claude-haiku-4-5-20251001)
Metric: Macro F1 (unweighted average across the three classes, to avoid the majority "neutral" class dominating the score)
Latency: wall-clock time per API/inference call, including network round-trip for hosted models
Cost: computed from each provider's published per-token pricing at the time of testing (September 2026); API pricing changes over time, so these figures should be treated as a snapshot, not a permanent benchmark
Evaluated: September 2026

Limitations

Narrow task scope. This adapter is specialized for short-form financial sentiment classification and is not a general-purpose instruction-following model. It should not be expected to perform well outside this task.
Small test set. 227 examples is sufficient to demonstrate the comparison methodology but is not large enough to make strong statistical claims about the true performance gap between models.
Frontier models were not fine-tuned. This is an intentional design choice (the fine-tuned model has an inherent advantage on its specific training distribution), not an oversight — but it means this is not a fair comparison of "best possible GPT/Claude performance," only "GPT/Claude used as a general-purpose classifier out of the box."
Gemini was not included. Testing was blocked by a Google Cloud billing-account verification failure unrelated to the model or code itself; this is not a statement about Gemini's actual performance on this task.
Pricing is a snapshot. Frontier API model names and pricing changed multiple times over the course of this project (weeks), and will likely have changed again by the time you're reading this — verify current pricing before drawing cost conclusions.
Hardware. See Hardware Requirements above — training and 4-bit inference require an NVIDIA GPU; macOS/Apple Silicon users can only run the unquantized fp16/bf16 path.

Reproducibility

Full training and evaluation code (data preparation, QLoRA training script, evaluation harness) is available on GitHub: afzall003/finllm-sentiment-qlora.

Citation

If you use this adapter, please cite the base model and the FinancialPhraseBank dataset:

text

@article{malo2014good,  title={Good debt or bad debt: Detecting semantic orientations in economic texts},  author={Malo, Pekka and Sinha, Ankur and Korhonen, Pekka and Wallenius, Jyrki and Takala, Pyry},  journal={Journal of the Association for Information Science and Technology},  year={2014}}

@article{malo2014good,  title={Good debt or bad debt: Detecting semantic orientations in economic texts},  author={Malo, Pekka and Sinha, Ankur and Korhonen, Pekka and Wallenius, Jyrki and Takala, Pyry},  journal={Journal of the Association for Information Science and Technology},  year={2014}}