Adorer0p

finkpi-qwen3-0.6b

Results

Whole test split: 2890 passages from 31 companies that appear in no other split, all filed after every training filing.

Table with columns: run, F1, 95% CI, P, R, schema ok, fires on empty, hard slice
run	F1	95% CI	P	R	schema ok	fires on empty	hard slice
finetuned-full	0.400	0.382–0.418	0.590	0.303	0.994	0.139	0.367
regex-full	0.356	0.343–0.370	0.320	0.400	1.000	0.455	0.333

Headline: micro F1 0.400 (95% CI 0.382–0.418) against a regex baseline of 0.356 (0.343–0.370). The intervals do not overlap.

The margin on F1 is real but modest. The larger differences are elsewhere:

Table with columns: fine-tune, regex baseline
	fine-tune	regex baseline
precision	0.590	0.320
recall	0.303	0.400
fires on a KPI-free passage	0.139	0.455

This model is the more trustworthy reader, not the more complete one. The baseline finds more of what is there by firing constantly — on 0.455 of passages containing no KPI at all. Recall is the model's standing weakness and the obvious target for future work.

What fine-tuning actually bought

Table with columns: stage, F1, schema valid, fires on empty, values off by a power of ten
stage	F1	schema valid	fires on empty	values off by a power of ten
base model, zero-shot	0.000	0.000	—	n/a: no parseable output
base model, 3-shot prompt	0.000	0.867	0.089	84% of near-misses
this adapter	0.400	0.994	0.139

Prompting fixed the format completely and the arithmetic not at all. Shown three worked examples, the base model emits well-formed JSON with the right field names and still reads $12,973 under an (in millions) header as 1,297,300,000. Learning that a table cell inherits the document's scale header is what the fine-tune contributes, and it is why F1 moves from 0.000 to 0.400. Scale errors in the run above: 2 of 2233 predictions.

Where the errors are

Table with columns: cause, count, share
cause	count	share
metric_not_in_passage	483	53%
value_other	205	22%
wrong_period_and_value	107	12%
wrong_basis	89	10%
wrong_period	27	3%
sign_flipped	2	0%

metric_not_in_passage is the largest bucket, and a hand audit of a sample found most of them are real line items the label aligner did not capture — verbatim spans pointing at genuine figures the gold set is missing. So the precision above is an underestimate of the model's true precision. It is reported unadjusted because adjusting it would require judging the model's output by hand, which is exactly what this harness exists to avoid.

Per metric

Table with columns: metric, P, R, F1, support
metric	P	R	F1	support
basic_eps	0.000	0.000	0.000	32
cash_and_equivalents	0.759	0.701	0.729	234
cost_of_revenue	0.345	0.164	0.222	116
diluted_eps

Metrics with a score of 0.000 are not usable. total_debt has essentially no support in the corpus and should be treated as unsupported rather than poor.

Usage

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "Qwen/Qwen3-0.6B"
tok = AutoTokenizer.from_pretrained(BASE)
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(BASE, device_map="auto"), "REPO_ID"
).merge_and_unload().eval()

SYSTEM = open("prompt_system.txt").read()   # shipped in this repo; use verbatim
msgs = [{"role": "system", "content": SYSTEM},
        {"role": "user", "content": f"Reporting period: Q3_2025\n\nPassage:\n{passage}"}]
ids = tok.apply_chat_template(msgs, return_tensors="pt", add_generation_prompt=True)
out = model.generate(ids.to(model.device), max_new_tokens=768, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

The prompt is part of the model. Change the system message and the numbers above no longer apply. max_new_tokens below ~768 truncates the longest answers: 2.2% of test passages need more than 320 tokens, and those hold 13.8% of all gold facts.

If you load in 4-bit, do not merge_and_unload(). Merging LoRA into NF4 weights means dequantise, add, requantise, and the requantisation error lands on exactly the weights the fine-tune changed.

Scope, and where it will fail

Trained on US-listed companies' English 8-K Item 2.02 exhibits. Expect degradation on non-US filers and IFRS presentations, segment and per-geography tables, guidance and forward-looking figures (excluded by construction), balance-sheet-heavy releases from financial institutions, and any metric outside the sixteen-item vocabulary.

Passage-level by design — it reads a window, not a whole release. For a document-level record, run every window and merge (finkpi.merge); agreement across overlapping windows is a usable confidence signal.

Not investment advice, and not a source of financial truth. It is a reading aid whose output should be checked against the filing. Every fact carries a verbatim span to make that check cheap; 0.972 of emitted spans are literal substrings of the input.

Training

QLoRA (4-bit NF4) on a single 4GB consumer GPU, peak 1.76 GB. 8806 examples, 1 epoch(s), rank 32, 4.6 hours. Prompt tokens masked; loss on the JSON answer only, and computed only at answer positions — full-sequence logits for this vocabulary are 594 MiB in fp32, larger than the quantised model, and do not fit.

Full configuration, losses and peak memory in train_metrics.json.

Reproducing

Everything, including the corpus build, is at the repository linked above. Per-run metrics for the baselines and this model are in eval_results/.

Provenance

json
{
  "git_commit": "",
  "python": "3.14.4",
  "platform": "Linux-6.18.33.2-microsoft-standard-WSL2-x86_64-with-glibc2.43",
  "torch": "2.13.0+cu130",
  "transformers": "4.57.6",
  "peft": "0.20.0",
  "gpu": {
    "name": "NVIDIA GeForce RTX 3050 Laptop GPU",
    "total_gb": 4.0
  },
  "prompt_version": "1.1.0",
  "seed": 20260822
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Adorer0p

Model Tree

Base

Qwen/Qwen3-0.6B

Adapter

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Results

Whole test split: 2890 passages from 31 companies that appear in no other split, all filed after every training filing.

Table with columns: run, F1, 95% CI, P, R, schema ok, fires on empty, hard slice
run	F1	95% CI	P	R	schema ok	fires on empty	hard slice
finetuned-full	0.400	0.382–0.418	0.590	0.303	0.994	0.139	0.367
regex-full	0.356	0.343–0.370	0.320	0.400	1.000	0.455	0.333

Headline: micro F1 0.400 (95% CI 0.382–0.418) against a regex baseline of 0.356 (0.343–0.370). The intervals do not overlap.

The margin on F1 is real but modest. The larger differences are elsewhere:

Table with columns: fine-tune, regex baseline
	fine-tune	regex baseline
precision	0.590	0.320
recall	0.303	0.400
fires on a KPI-free passage	0.139	0.455

What fine-tuning actually bought

Table with columns: stage, F1, schema valid, fires on empty, values off by a power of ten
stage	F1	schema valid	fires on empty	values off by a power of ten
base model, zero-shot	0.000	0.000	—	n/a: no parseable output
base model, 3-shot prompt	0.000	0.867	0.089	84% of near-misses
this adapter	0.400	0.994	0.139

Where the errors are

Table with columns: cause, count, share
cause	count	share
metric_not_in_passage	483	53%
value_other	205	22%
wrong_period_and_value	107	12%
wrong_basis	89	10%
wrong_period	27	3%
sign_flipped	2	0%

Per metric

Table with columns: metric, P, R, F1, support
metric	P	R	F1	support
basic_eps	0.000	0.000	0.000	32
cash_and_equivalents	0.759	0.701	0.729	234
cost_of_revenue	0.345	0.164	0.222	116
diluted_eps

Metrics with a score of 0.000 are not usable. total_debt has essentially no support in the corpus and should be treated as unsupported rather than poor.

Usage

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "Qwen/Qwen3-0.6B"
tok = AutoTokenizer.from_pretrained(BASE)
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(BASE, device_map="auto"), "REPO_ID"
).merge_and_unload().eval()

SYSTEM = open("prompt_system.txt").read()   # shipped in this repo; use verbatim
msgs = [{"role": "system", "content": SYSTEM},
        {"role": "user", "content": f"Reporting period: Q3_2025\n\nPassage:\n{passage}"}]
ids = tok.apply_chat_template(msgs, return_tensors="pt", add_generation_prompt=True)
out = model.generate(ids.to(model.device), max_new_tokens=768, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

Scope, and where it will fail

Training

Full configuration, losses and peak memory in train_metrics.json.

Reproducing

Everything, including the corpus build, is at the repository linked above. Per-run metrics for the baselines and this model are in eval_results/.

Provenance

json
{
  "git_commit": "",
  "python": "3.14.4",
  "platform": "Linux-6.18.33.2-microsoft-standard-WSL2-x86_64-with-glibc2.43",
  "torch": "2.13.0+cu130",
  "transformers": "4.57.6",
  "peft": "0.20.0",
  "gpu": {
    "name": "NVIDIA GeForce RTX 3050 Laptop GPU",
    "total_gb": 4.0
  },
  "prompt_version": "1.1.0",
  "seed": 20260822
}

finkpi-qwen3-0.6b

README

Results

What fine-tuning actually bought

Where the errors are

Per metric

Usage

Scope, and where it will fail

Training

Reproducing

Provenance

Explore FriendliAI today

README

Results

What fine-tuning actually bought

Where the errors are

Per metric

Usage

Scope, and where it will fail

Training

Reproducing

Provenance