fotapol

qwen3-1.7b-financial-rag-lora-v3

README

License: apache-2.0

Intended use

The intended input is a grounded chat prompt containing:

a system instruction requiring context-only answers;
up to five retrieved financial-document chunks labeled [Source N];
one user question.

The intended output is a short answer with units and source citations, or exactly:

markdown
I cannot answer this question from the supplied context.

This adapter is intended for demonstrations, learning, and experimentation. It is not suitable for autonomous financial, investment, legal, tax, or accounting decisions.

Loading the adapter

Install compatible versions:

bash
pip install "transformers>=4.57.6" "peft>=0.20.0" accelerate torch

Load the pinned base model and unmerged adapter:

python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE_MODEL_ID = "Qwen/Qwen3-1.7B"
BASE_MODEL_REVISION = "70d244cc86ccca08cf5af4e1e306ecf908b1ad5e"
ADAPTER_ID = "fotapol/qwen3-1.7b-financial-rag-lora-v3"

tokenizer = AutoTokenizer.from_pretrained(ADAPTER_ID)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL_ID,
    revision=BASE_MODEL_REVISION,
    dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_ID,
    is_trainable=False,
)
model.eval()

messages = [
    {
        "role": "system",
        "content": (
            "You are a financial document question-answering assistant. "
            "Use only the retrieved context supplied by the user. "
            "If the context does not fully support an answer, respond exactly: "
            "I cannot answer this question from the supplied context. "
            "When an answer is supported, cite the relevant source labels."
        ),
    },
    {
        "role": "user",
        "content": (
            "Retrieved context:\n\n"
            "[Source 1 | page 4 | chunk_id example-1]\n"
            "Revenue was $14.1 million in 2025 and $12.4 million in 2024.\n\n"
            "Question:\n"
            "By how much did revenue increase from 2024 to 2025?"
        ),
    },
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.inference_mode():
    generated = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
        pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

answer = tokenizer.decode(
    generated[0, inputs["input_ids"].shape[1] :],
    skip_special_tokens=True,
).strip()
print(answer)

The deterministic settings above reproduce the project evaluation configuration. They are not a claim that greedy decoding is optimal for every Qwen3 use case.

Training

Method: 4-bit NF4 QLoRA
LoRA rank: 16
LoRA alpha: 32
LoRA dropout: 0.05
Target modules: q, k, v, o, gate, up, and down projections
Epochs: 1
Optimizer steps: 1,460
Effective batch size: 8
Maximum sequence length: 4,096
Thinking mode: disabled
Loss: assistant tokens only
Training records: 11,674
Validation records: 1,493
Test records reserved but not evaluated: 1,475

Training examples were derived from FinQA and DocFinQA and converted into production-shaped RAG conversations using frozen BM25+dense RRF contexts, source labels, units, visible calculations, and synthetic insufficient-context refusals. Report-level splitting prevents documents from crossing train, validation, and test.

See dataset_manifest.json and training_metadata.json for the exact configuration and counts.

Validation results

The complete 1,493-example schema-v5 validation split was evaluated with identical frozen prompts for the pinned base and adapter. Both conditions used thinking disabled, greedy decoding, a 4,096-token input limit, and a 128-token output limit.

Table with columns: Metric, Pinned base, Adapter v3, Difference
Metric	Pinned base	Adapter v3	Difference
Overall accuracy	16.28%	30.48%	+14.20 pp
Final-value accuracy	3.19%	23.70%	+20.50 pp
Unit accuracy	16.19%	89.18%	+72.99 pp
Citation completion	5.80%	75.80%	+70.00 pp

These are project-specific validation metrics, not public benchmark or leaderboard results. The held-out schema-v5 test split was not run for this MVP release. Exact evaluation configuration is stored in evaluation/validation_metrics.json.

Limitations

The adapter is not a reliable calculator. It often emits a sensible calculation structure but an incorrect final arithmetic result.
Final-value accuracy is 23.70%; important figures must be checked against the cited document.
Accuracy declines substantially as retrieved context becomes longer.
Questions requiring evidence from three or more sources remain difficult.
Citation labels are usually valid, but the model sometimes omits part of the required evidence.
Unsupported refusal behavior is uneven: it is stronger on DocFinQA-shaped inputs and weaker on FinQA-shaped inputs.
The synthetic refusal rule requires every gold source-lineage element. Some contexts labeled insufficient still contain plausible alternative evidence, so a generated answer is not always a simple hallucination.
Training and evaluation cover English financial-report questions and do not establish performance in other domains or languages.

Always display retrieved context, page information, and chunk IDs beside generated answers.

Reproducibility

Training manifest SHA-256: 9ba6699401ef30cd7646cb0e86f15b88afa904b9232db5e1d75d74f162a3bec9
Adapter weights SHA-256: dcafae02812dbb479202bde0ffbe7758bf93380570775c4fe132d91b80136d8b
Validation evaluation archive SHA-256: 842da65320a8c6f9c6936215684368c11c9bb03116d372f7b4755db3a9f6010d

License

The adapter follows the Apache-2.0 license of the Qwen3-1.7B base model. Users are responsible for reviewing the licenses and terms of the base model and source datasets.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

fotapol

Model Tree

Base

Qwen/Qwen3-1.7B

Adapter

this model

Input Modalities

Text

Output Modalities