privoid-extract-qwen3-4b-lora API & Inference Endpoint

🧾 What it returns

text

Nordhafen Workshop Supplies GmbH & Co. KG236 Hafenstrasse 20457 Hamburg…TAX INVOICEInvoice Number:IN0089266Date:30.04.2026Due Date:29.06.2026Terms:60 daysBILL TO…Ostwall Trucking GmbH…Brake pads, trailer axle set11.151,181.151,18…Subtotal5.632,69 €MwSt 19%1.070,21 €TOTAL6.702,90 €

json

{  "schema_version": "invoice-v0.1",  "document_type": "invoice",  "status": "complete",  "invoice_id": {"raw": "IN0089266", "evidence": "IN0089266"},  "issue_date": {"raw": "30.04.2026", "normalized": "2026-04-30",                 "evidence": "30.04.2026"},  "due_date":   {"raw": "29.06.2026", "normalized": "2026-06-29",                 "evidence": "29.06.2026"},  "currency":   {"raw": "€", "code": "EUR", "evidence": "6.702,90 €"},  "supplier_name": {"raw": "Nordhafen Workshop Supplies GmbH & Co. KG", "evidence": "…"},  "buyer_name":    {"raw": "Ostwall Trucking GmbH", "evidence": "…"},  "purchase_order_ref": null,  "subtotal":    {"raw": "5.632,69", "decimal": "5632.69", "evidence": "5.632,69 €"},  "tax_total":   {"raw": "1.070,21", "decimal": "1070.21", "evidence": "1.070,21 €"},  "grand_total": {"raw": "6.702,90", "decimal": "6702.90", "evidence": "6.702,90 €"},  "line_items": [    {"description": {"raw": "Brake pads, trailer axle set", "evidence": "…"},     "quantity":    {"raw": "1", "evidence": "1"},     "unit_price":  {"raw": "1.151,18", "decimal": "1151.18", "evidence": "1.151,18"},     "line_total":  {"raw": "1.151,18", "decimal": "1151.18", "evidence": "1.151,18"}}  ]}

Real pypdf text and real model output for benchmark document clean-de-grid-eu-decimals. Lines marked … are omitted here.

Field types

Table with columns: Kind, Shape, Example
Kind	Shape	Example
Text	`{raw, evidence}`	invoice number, supplier, buyer, PO reference, line description
Date	`{raw, normalized, evidence}`	`normalized` is `YYYY-MM-DD` or `null`
Currency	`{raw, code, evidence}`	`code` is ISO 4217 or `null`
Money	`{raw, decimal, evidence}`	`decimal` has no symbol or thousands separator and uses `.` as the decimal point
Status	`complete` · `partial` · `unsupported`	non-invoices are `unsupported` with every field `null`

When it normalizes, and when it refuses to guess

Table with columns: On the document, normalized / code
On the document	`normalized` / `code`
`2026-04-30`, `30 April 2026`, `30.04.2026` (day > 12)	✅ `2026-04-30`
`03/04/2026`, supplier country printed as United States	✅ `2026-03-04` (month first)
`03/04/2026`, another date on the page like `03/28/2026`	✅ the order is settled by the other date
`03/04/2026`, nothing on the page settles the order	⬜ `null` (ambiguous)
`€`, `£`, `N$`, or a printed ISO code such as `ZAR`	✅ `EUR`, `GBP`, `NAD`, `ZAR`
`R` + "South Africa" printed · `$` + "United States" printed	✅ `ZAR` · `USD`
`$`, `R` or `K` with no country printed	⬜ `null`

The model never calculates a missing value. If there is no due date, only "Net 30", the due date is null.

🚀 Quickstart

bash

pip install unsloth jsonschema pypdf

python

import jsonfrom huggingface_hub import hf_hub_downloadfrom unsloth import FastLanguageModel REPO = "Privoid-ai/privoid-extract-qwen3-4b-lora"SYSTEM = open(hf_hub_download(REPO, "system_prompt.txt"), encoding="utf-8").read().strip()ASSISTANT_PREFIX = "<think>\n\n</think>\n\n"   # the chat template starts every assistant turn with this model, tokenizer = FastLanguageModel.from_pretrained(REPO, max_seq_length=8192, load_in_4bit=True)FastLanguageModel.for_inference(model) def extract(invoice_text: str, max_new_tokens: int = 3072) -> dict:    messages = [        {"role": "system", "content": SYSTEM},        {"role": "user", "content": "Invoice text:\n<document>\n" + invoice_text + "\n</document>"},    ]    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + ASSISTANT_PREFIX    batch = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)    out = model.generate(**batch, max_new_tokens=max_new_tokens, do_sample=False, pad_token_id=tokenizer.eos_token_id)    return json.loads(tokenizer.decode(out[0, batch["input_ids"].shape[1]:], skip_special_tokens=True))

The adapter was trained with the exact system_prompt.txt and the empty think block. Change either and quality drops.

python

from pypdf import PdfReader def pdf_text(path: str) -> str:    # Pages are joined the same way as in the benchmark evaluation.    return "\n\f\n".join(page.extract_text() or "" for page in PdfReader(path).pages) text = pdf_text("invoice.pdf")if len(text.strip()) < 80:    raise ValueError("No usable text layer: run OCR first (this model reads text, not images).")result = extract(text)print(result["status"], result["grand_total"])

python

import jsonfrom jsonschema import Draft202012Validator validator = Draft202012Validator(json.load(open("invoice-v0.1.schema.json", encoding="utf-8"))) def problems(result: dict, text: str) -> list[str]:    """Schema errors, plus every evidence that is not an exact substring of the text (or does not contain raw)."""    found = [f"schema: {e.message}" for e in validator.iter_errors(result)]    def walk(node, path):        if isinstance(node, dict):            if "evidence" in node:                ev, raw = node.get("evidence"), node.get("raw")                if not isinstance(ev, str) or ev not in text: found.append(f"{path}: evidence not in document")                elif "raw" in node and (not isinstance(raw, str) or raw not in ev): found.append(f"{path}: raw not in evidence")            for key, value in node.items(): walk(value, f"{path}.{key}")        elif isinstance(node, list):            for i, value in enumerate(node): walk(value, f"{path}[{i}]")    walk(result, "$")    return found issues = problems(result, text)print("accept" if not issues else f"send to review: {issues[:3]}")

On the 99 benchmark invoices this check passes 75 outputs unchanged, matching the 75.8% reported below. Route the rest to a person.

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel BASE = "unsloth/Qwen3-4B-Instruct-2507-unsloth-bnb-4bit"tokenizer = AutoTokenizer.from_pretrained(REPO)base = AutoModelForCausalLM.from_pretrained(BASE, revision="7744afa8566e264af1a92a806d8d9aae00cc7c78",                                            quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16),                                            device_map="auto")model = PeftModel.from_pretrained(base, REPO).eval()# Then build the prompt exactly as in snippet 1 (system prompt + ASSISTANT_PREFIX) and generate greedily.

This path has not been benchmarked. All results on this card come from snippet 1.

📊 Benchmark

All three models used the same chat template, greedy decoding, the same machine and no output repair. Every number and every field-level failure is in eval/.

Data. drew-ipp/invoice-extraction-benchmark, revision 446bab0, 99 invoice PDFs read through the pypdf text layer. None of it was used for training.
Scoring. A prediction is compared using normalized, then code, then decimal, then raw, against ISO-formatted gold labels.

Table with columns: Metric, Prompted base, v1 adapter, v2 adapter
Metric	Prompted base	v1 adapter	v2 adapter
Header field accuracy (micro, 8 fields)	86.3%	60.7%	87.4%
All 8 header fields correct (per doc)	75.8%	1.0%	68.7%
Raw JSON validity	89.9%	88.9%	91.9%
Schema + evidence validity	0.0%	30.3%	75.8%
Median latency (RTX 3060)	33.4 s	38.5 s	41.1 s

[!NOTE] The base model was scored with the v1 prompt. v2 adds the date and currency rules above, and the base was not re-run with it. The base scores 0% on schema validity because it writes plain strings where the schema requires objects.

By field

Table with columns: Field, Prompted base, v1, v2, Scored
Field	Prompted base	v1	v2	Scored
Vendor name	89	88	91	99
Invoice number	86	83	89	97
Invoice date	89	16	91	99
Due date	62	6	65	73
Currency	84	23	74	99
Subtotal	80	78	81	99
Tax	83	85	86	99
Total	86	85	91	99

By document category

Table with columns: Category, Docs, Prompted base, v2
Category	Docs	Prompted base	v2
freight	16	99.2%	100%
hard	9	100%	100%
tax	14	91.8%	94.5%
currency	19	90.0%	92.0%
clean	31	96.5%	93.9%
multipage (3–8 pages)	10	10.0%	20.0%

Leaving out the 10 multi-page documents, header accuracy is 95.3% for v2 against 95.2% for the base.

Non-invoices

Table with columns: Kind, n, Prompted base, v2
Kind	n	Prompted base	v2
Credit note	10	0 rejected (3 complete, 7 partial)	10 unsupported
Multiple invoices in one file	2	1 rejected	2 unsupported
Statement	2	2 rejected	2 unsupported

Held-out synthetic test set (60 documents)

The suppliers and layout families in this set never appear in training.

Table with columns: Raw JSON, Schema + evidence, All critical fields exact, All line items exact, Median latency
Raw JSON	Schema + evidence	All critical fields exact	All line items exact	Median latency
100%	98.3%	90.0%	96.7%	35.1 s

Where v2 still misses

Table with columns: Failure, Count, Why, What to do
Failure	Count	Why	What to do
`$` on US invoices with no country printed	13 currency	The rule says not to infer currency from `$` alone	Map `$` from your own supplier data
Multi-page invoices with very many lines	8 docs	Output runs past 3,072 tokens partway through `line_items`	Raise `max_new_tokens`, or split the document
Tax-inclusive and dual-currency layouts	10 subtotal, 4 currency	The model copies the gross or foreign-currency column	Review these layouts by hand

🏋️ Training

Table

Data	600 synthetic invoices. No real or customer documents.
	500 generated programmatically across 10 supplier countries (US, GB, IE, DE, FR, NL, ZA, NA, ZM, AU): locale number formats, 7 date styles, currency symbols and codes, freight, discounts, missing tax rows, 2-page invoices with 10–22 lines, prompt-injection footers and 25 non-invoice negatives.
	100 re-rendered into varied layouts by a local `gpt-oss-20b` teacher from programmatic facts, then reviewed.
Labels	Generated from the facts, never by a model. Each one passes the schema and exact-evidence checks.
Splits	Train 600 · dev 40 (20 used for eval loss) · test 60. Supplier names and layout families are disjoint across splits.
LoRA	r = 16, α = 32, dropout 0, on `q,k,v,o,gate,up,down` projections
Optimizer	AdamW 8-bit, lr 2e-4, linear schedule, 5 warm-up steps, seed 42
Batch	1 × 4 gradient accumulation · up to 4,096 tokens, no truncation · assistant-only loss
Run	105 steps (early stopping, best checkpoint kept) · 32 min on an RTX 3060 12 GB
Memory	Peak 23.7 GiB reserved, so training spilled into shared system memory. Inference fits in 12 GB.

v1 was trained for 40 steps on simpler data. That data labelled every non-ISO date as un-normalized and contained no currency symbols, thousands separators or long invoices. On the benchmark, v1 normalized only 16 of 99 invoice dates and lost to the prompted base (60.7% vs 86.3%). v2 replaced the training data, not the method. The benchmark was used for evaluation only, once per version.

⚠️ Limitations

Synthetic training only. Real invoices bring messier layouts, OCR noise and conventions not covered here. Measure on your own documents first.
Long documents. Invoices whose JSON needs more than about 3,000 output tokens are cut off. Inputs above about 5,000 tokens come close to the memory limit on 12 GB.
Strict on purpose. The date and currency rules leave null where a person might infer from context, such as a US street address without "USA".
Text only, English only. Scanned PDFs need OCR first.
Tax-inclusive and multi-currency invoices can produce a subtotal from the wrong figure.
Untrusted input. The model is trained to ignore instructions inside documents, but that is not a security boundary. Always run the validation in snippet 3.
Benchmark caveat. The benchmark comes from InvoiceParser Pro, a vendor, not an independent body. It is small and partly synthetic.
Not for unattended financial, tax or legal decisions. Keep a person in the loop.

Intended use: research and prototyping of private, on-device invoice extraction where every value must be traceable, feeding a deterministic validator and human review.

📁 Files

Table with columns: File, Purpose
File	Purpose
`adapter_model.safetensors`	LoRA weights, 132 MB, sha256 `e354a351…850ce83a`
`adapter_config.json`	PEFT config, base pinned to revision `7744afa`
`system_prompt.txt`	The exact system prompt used in training and evaluation
`tokenizer.json` · `tokenizer_config.json` · `chat_template.jinja`	Tokenizer and chat template
`invoice-v0.1.schema.json`	Output JSON Schema
`eval/release_metrics_v2.json`	v2 metrics, statuses and every field-level failure
`eval/release_metrics_v1_and_base.json`	v1 and prompted-base metrics
`assets/`	Charts on this card

📜 License and attribution

Adapter: Apache-2.0.
Base model: Qwen3-4B-Instruct-2507, Qwen team, Apache-2.0. 4-bit checkpoint by Unsloth.
Teacher model for layout re-rendering: gpt-oss-20b, OpenAI, Apache-2.0.
Evaluation data: invoice-extraction-benchmark by InvoiceParser Pro, CC BY 4.0, used for evaluation only.

📝 Citation

bibtex

@misc{privoid_extract_v2_2026,  title        = {Privoid Extract v2: Evidence-Grounded Invoice Extraction with a Qwen3-4B LoRA},  year         = {2026},  howpublished = {\url{https://huggingface.co/Privoid-ai/privoid-extract-qwen3-4b-lora}},  note         = {LoRA adapter for Qwen3-4B-Instruct-2507}}

🧾 What it returns

text

Nordhafen Workshop Supplies GmbH & Co. KG236 Hafenstrasse 20457 Hamburg…TAX INVOICEInvoice Number:IN0089266Date:30.04.2026Due Date:29.06.2026Terms:60 daysBILL TO…Ostwall Trucking GmbH…Brake pads, trailer axle set11.151,181.151,18…Subtotal5.632,69 €MwSt 19%1.070,21 €TOTAL6.702,90 €

Nordhafen Workshop Supplies GmbH & Co. KG236 Hafenstrasse 20457 Hamburg…TAX INVOICEInvoice Number:IN0089266Date:30.04.2026Due Date:29.06.2026Terms:60 daysBILL TO…Ostwall Trucking GmbH…Brake pads, trailer axle set11.151,181.151,18…Subtotal5.632,69 €MwSt 19%1.070,21 €TOTAL6.702,90 €

json

{  "schema_version": "invoice-v0.1",  "document_type": "invoice",  "status": "complete",  "invoice_id": {"raw": "IN0089266", "evidence": "IN0089266"},  "issue_date": {"raw": "30.04.2026", "normalized": "2026-04-30",                 "evidence": "30.04.2026"},  "due_date":   {"raw": "29.06.2026", "normalized": "2026-06-29",                 "evidence": "29.06.2026"},  "currency":   {"raw": "€", "code": "EUR", "evidence": "6.702,90 €"},  "supplier_name": {"raw": "Nordhafen Workshop Supplies GmbH & Co. KG", "evidence": "…"},  "buyer_name":    {"raw": "Ostwall Trucking GmbH", "evidence": "…"},  "purchase_order_ref": null,  "subtotal":    {"raw": "5.632,69", "decimal": "5632.69", "evidence": "5.632,69 €"},  "tax_total":   {"raw": "1.070,21", "decimal": "1070.21", "evidence": "1.070,21 €"},  "grand_total": {"raw": "6.702,90", "decimal": "6702.90", "evidence": "6.702,90 €"},  "line_items": [    {"description": {"raw": "Brake pads, trailer axle set", "evidence": "…"},     "quantity":    {"raw": "1", "evidence": "1"},     "unit_price":  {"raw": "1.151,18", "decimal": "1151.18", "evidence": "1.151,18"},     "line_total":  {"raw": "1.151,18", "decimal": "1151.18", "evidence": "1.151,18"}}  ]}

{  "schema_version": "invoice-v0.1",  "document_type": "invoice",  "status": "complete",  "invoice_id": {"raw": "IN0089266", "evidence": "IN0089266"},  "issue_date": {"raw": "30.04.2026", "normalized": "2026-04-30",                 "evidence": "30.04.2026"},  "due_date":   {"raw": "29.06.2026", "normalized": "2026-06-29",                 "evidence": "29.06.2026"},  "currency":   {"raw": "€", "code": "EUR", "evidence": "6.702,90 €"},  "supplier_name": {"raw": "Nordhafen Workshop Supplies GmbH & Co. KG", "evidence": "…"},  "buyer_name":    {"raw": "Ostwall Trucking GmbH", "evidence": "…"},  "purchase_order_ref": null,  "subtotal":    {"raw": "5.632,69", "decimal": "5632.69", "evidence": "5.632,69 €"},  "tax_total":   {"raw": "1.070,21", "decimal": "1070.21", "evidence": "1.070,21 €"},  "grand_total": {"raw": "6.702,90", "decimal": "6702.90", "evidence": "6.702,90 €"},  "line_items": [    {"description": {"raw": "Brake pads, trailer axle set", "evidence": "…"},     "quantity":    {"raw": "1", "evidence": "1"},     "unit_price":  {"raw": "1.151,18", "decimal": "1151.18", "evidence": "1.151,18"},     "line_total":  {"raw": "1.151,18", "decimal": "1151.18", "evidence": "1.151,18"}}  ]}

Real pypdf text and real model output for benchmark document clean-de-grid-eu-decimals. Lines marked … are omitted here.

Field types

Table with columns: Kind, Shape, Example
Kind	Shape	Example
Text	`{raw, evidence}`	invoice number, supplier, buyer, PO reference, line description
Date	`{raw, normalized, evidence}`	`normalized` is `YYYY-MM-DD` or `null`
Currency	`{raw, code, evidence}`	`code` is ISO 4217 or `null`
Money	`{raw, decimal, evidence}`	`decimal` has no symbol or thousands separator and uses `.` as the decimal point
Status	`complete` · `partial` · `unsupported`	non-invoices are `unsupported` with every field `null`

When it normalizes, and when it refuses to guess

Table with columns: On the document, normalized / code
On the document	`normalized` / `code`
`2026-04-30`, `30 April 2026`, `30.04.2026` (day > 12)	✅ `2026-04-30`
`03/04/2026`, supplier country printed as United States	✅ `2026-03-04` (month first)
`03/04/2026`, another date on the page like `03/28/2026`	✅ the order is settled by the other date
`03/04/2026`, nothing on the page settles the order	⬜ `null` (ambiguous)
`€`, `£`, `N$`, or a printed ISO code such as `ZAR`	✅ `EUR`, `GBP`, `NAD`, `ZAR`
`R` + "South Africa" printed · `$` + "United States" printed	✅ `ZAR` · `USD`
`$`, `R` or `K` with no country printed	⬜ `null`

The model never calculates a missing value. If there is no due date, only "Net 30", the due date is null.

🚀 Quickstart

bash

pip install unsloth jsonschema pypdf

python

import jsonfrom huggingface_hub import hf_hub_downloadfrom unsloth import FastLanguageModel REPO = "Privoid-ai/privoid-extract-qwen3-4b-lora"SYSTEM = open(hf_hub_download(REPO, "system_prompt.txt"), encoding="utf-8").read().strip()ASSISTANT_PREFIX = "<think>\n\n</think>\n\n"   # the chat template starts every assistant turn with this model, tokenizer = FastLanguageModel.from_pretrained(REPO, max_seq_length=8192, load_in_4bit=True)FastLanguageModel.for_inference(model) def extract(invoice_text: str, max_new_tokens: int = 3072) -> dict:    messages = [        {"role": "system", "content": SYSTEM},        {"role": "user", "content": "Invoice text:\n<document>\n" + invoice_text + "\n</document>"},    ]    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + ASSISTANT_PREFIX    batch = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)    out = model.generate(**batch, max_new_tokens=max_new_tokens, do_sample=False, pad_token_id=tokenizer.eos_token_id)    return json.loads(tokenizer.decode(out[0, batch["input_ids"].shape[1]:], skip_special_tokens=True))

import jsonfrom huggingface_hub import hf_hub_downloadfrom unsloth import FastLanguageModel REPO = "Privoid-ai/privoid-extract-qwen3-4b-lora"SYSTEM = open(hf_hub_download(REPO, "system_prompt.txt"), encoding="utf-8").read().strip()ASSISTANT_PREFIX = "<think>\n\n</think>\n\n"   # the chat template starts every assistant turn with this model, tokenizer = FastLanguageModel.from_pretrained(REPO, max_seq_length=8192, load_in_4bit=True)FastLanguageModel.for_inference(model) def extract(invoice_text: str, max_new_tokens: int = 3072) -> dict:    messages = [        {"role": "system", "content": SYSTEM},        {"role": "user", "content": "Invoice text:\n<document>\n" + invoice_text + "\n</document>"},    ]    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + ASSISTANT_PREFIX    batch = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)    out = model.generate(**batch, max_new_tokens=max_new_tokens, do_sample=False, pad_token_id=tokenizer.eos_token_id)    return json.loads(tokenizer.decode(out[0, batch["input_ids"].shape[1]:], skip_special_tokens=True))

The adapter was trained with the exact system_prompt.txt and the empty think block. Change either and quality drops.

python

from pypdf import PdfReader def pdf_text(path: str) -> str:    # Pages are joined the same way as in the benchmark evaluation.    return "\n\f\n".join(page.extract_text() or "" for page in PdfReader(path).pages) text = pdf_text("invoice.pdf")if len(text.strip()) < 80:    raise ValueError("No usable text layer: run OCR first (this model reads text, not images).")result = extract(text)print(result["status"], result["grand_total"])

from pypdf import PdfReader def pdf_text(path: str) -> str:    # Pages are joined the same way as in the benchmark evaluation.    return "\n\f\n".join(page.extract_text() or "" for page in PdfReader(path).pages) text = pdf_text("invoice.pdf")if len(text.strip()) < 80:    raise ValueError("No usable text layer: run OCR first (this model reads text, not images).")result = extract(text)print(result["status"], result["grand_total"])

python

import jsonfrom jsonschema import Draft202012Validator validator = Draft202012Validator(json.load(open("invoice-v0.1.schema.json", encoding="utf-8"))) def problems(result: dict, text: str) -> list[str]:    """Schema errors, plus every evidence that is not an exact substring of the text (or does not contain raw)."""    found = [f"schema: {e.message}" for e in validator.iter_errors(result)]    def walk(node, path):        if isinstance(node, dict):            if "evidence" in node:                ev, raw = node.get("evidence"), node.get("raw")                if not isinstance(ev, str) or ev not in text: found.append(f"{path}: evidence not in document")                elif "raw" in node and (not isinstance(raw, str) or raw not in ev): found.append(f"{path}: raw not in evidence")            for key, value in node.items(): walk(value, f"{path}.{key}")        elif isinstance(node, list):            for i, value in enumerate(node): walk(value, f"{path}[{i}]")    walk(result, "$")    return found issues = problems(result, text)print("accept" if not issues else f"send to review: {issues[:3]}")

import jsonfrom jsonschema import Draft202012Validator validator = Draft202012Validator(json.load(open("invoice-v0.1.schema.json", encoding="utf-8"))) def problems(result: dict, text: str) -> list[str]:    """Schema errors, plus every evidence that is not an exact substring of the text (or does not contain raw)."""    found = [f"schema: {e.message}" for e in validator.iter_errors(result)]    def walk(node, path):        if isinstance(node, dict):            if "evidence" in node:                ev, raw = node.get("evidence"), node.get("raw")                if not isinstance(ev, str) or ev not in text: found.append(f"{path}: evidence not in document")                elif "raw" in node and (not isinstance(raw, str) or raw not in ev): found.append(f"{path}: raw not in evidence")            for key, value in node.items(): walk(value, f"{path}.{key}")        elif isinstance(node, list):            for i, value in enumerate(node): walk(value, f"{path}[{i}]")    walk(result, "$")    return found issues = problems(result, text)print("accept" if not issues else f"send to review: {issues[:3]}")

On the 99 benchmark invoices this check passes 75 outputs unchanged, matching the 75.8% reported below. Route the rest to a person.

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel BASE = "unsloth/Qwen3-4B-Instruct-2507-unsloth-bnb-4bit"tokenizer = AutoTokenizer.from_pretrained(REPO)base = AutoModelForCausalLM.from_pretrained(BASE, revision="7744afa8566e264af1a92a806d8d9aae00cc7c78",                                            quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16),                                            device_map="auto")model = PeftModel.from_pretrained(base, REPO).eval()# Then build the prompt exactly as in snippet 1 (system prompt + ASSISTANT_PREFIX) and generate greedily.

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel BASE = "unsloth/Qwen3-4B-Instruct-2507-unsloth-bnb-4bit"tokenizer = AutoTokenizer.from_pretrained(REPO)base = AutoModelForCausalLM.from_pretrained(BASE, revision="7744afa8566e264af1a92a806d8d9aae00cc7c78",                                            quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16),                                            device_map="auto")model = PeftModel.from_pretrained(base, REPO).eval()# Then build the prompt exactly as in snippet 1 (system prompt + ASSISTANT_PREFIX) and generate greedily.

This path has not been benchmarked. All results on this card come from snippet 1.

📊 Benchmark

All three models used the same chat template, greedy decoding, the same machine and no output repair. Every number and every field-level failure is in eval/.

Data. drew-ipp/invoice-extraction-benchmark, revision 446bab0, 99 invoice PDFs read through the pypdf text layer. None of it was used for training.
Scoring. A prediction is compared using normalized, then code, then decimal, then raw, against ISO-formatted gold labels.

Table with columns: Metric, Prompted base, v1 adapter, v2 adapter
Metric	Prompted base	v1 adapter	v2 adapter
Header field accuracy (micro, 8 fields)	86.3%	60.7%	87.4%
All 8 header fields correct (per doc)	75.8%	1.0%	68.7%
Raw JSON validity	89.9%	88.9%	91.9%
Schema + evidence validity	0.0%	30.3%	75.8%
Median latency (RTX 3060)	33.4 s	38.5 s	41.1 s

[!NOTE] The base model was scored with the v1 prompt. v2 adds the date and currency rules above, and the base was not re-run with it. The base scores 0% on schema validity because it writes plain strings where the schema requires objects.

By field

Table with columns: Field, Prompted base, v1, v2, Scored
Field	Prompted base	v1	v2	Scored
Vendor name	89	88	91	99
Invoice number	86	83	89	97
Invoice date	89	16	91	99
Due date	62	6	65	73
Currency	84	23	74	99
Subtotal	80	78	81	99
Tax	83	85	86	99
Total	86	85	91	99

By document category

Table with columns: Category, Docs, Prompted base, v2
Category	Docs	Prompted base	v2
freight	16	99.2%	100%
hard	9	100%	100%
tax	14	91.8%	94.5%
currency	19	90.0%	92.0%
clean	31	96.5%	93.9%
multipage (3–8 pages)	10	10.0%	20.0%

Leaving out the 10 multi-page documents, header accuracy is 95.3% for v2 against 95.2% for the base.

Non-invoices

Table with columns: Kind, n, Prompted base, v2
Kind	n	Prompted base	v2
Credit note	10	0 rejected (3 complete, 7 partial)	10 unsupported
Multiple invoices in one file	2	1 rejected	2 unsupported
Statement	2	2 rejected	2 unsupported

Held-out synthetic test set (60 documents)

The suppliers and layout families in this set never appear in training.

Table with columns: Raw JSON, Schema + evidence, All critical fields exact, All line items exact, Median latency
Raw JSON	Schema + evidence	All critical fields exact	All line items exact	Median latency
100%	98.3%	90.0%	96.7%	35.1 s

Where v2 still misses

Table with columns: Failure, Count, Why, What to do
Failure	Count	Why	What to do
`$` on US invoices with no country printed	13 currency	The rule says not to infer currency from `$` alone	Map `$` from your own supplier data
Multi-page invoices with very many lines	8 docs	Output runs past 3,072 tokens partway through `line_items`	Raise `max_new_tokens`, or split the document
Tax-inclusive and dual-currency layouts	10 subtotal, 4 currency	The model copies the gross or foreign-currency column	Review these layouts by hand

🏋️ Training

Table

Data	600 synthetic invoices. No real or customer documents.
	500 generated programmatically across 10 supplier countries (US, GB, IE, DE, FR, NL, ZA, NA, ZM, AU): locale number formats, 7 date styles, currency symbols and codes, freight, discounts, missing tax rows, 2-page invoices with 10–22 lines, prompt-injection footers and 25 non-invoice negatives.
	100 re-rendered into varied layouts by a local `gpt-oss-20b` teacher from programmatic facts, then reviewed.
Labels	Generated from the facts, never by a model. Each one passes the schema and exact-evidence checks.
Splits	Train 600 · dev 40 (20 used for eval loss) · test 60. Supplier names and layout families are disjoint across splits.
LoRA	r = 16, α = 32, dropout 0, on `q,k,v,o,gate,up,down` projections
Optimizer	AdamW 8-bit, lr 2e-4, linear schedule, 5 warm-up steps, seed 42
Batch	1 × 4 gradient accumulation · up to 4,096 tokens, no truncation · assistant-only loss
Run	105 steps (early stopping, best checkpoint kept) · 32 min on an RTX 3060 12 GB
Memory	Peak 23.7 GiB reserved, so training spilled into shared system memory. Inference fits in 12 GB.

⚠️ Limitations

Synthetic training only. Real invoices bring messier layouts, OCR noise and conventions not covered here. Measure on your own documents first.
Long documents. Invoices whose JSON needs more than about 3,000 output tokens are cut off. Inputs above about 5,000 tokens come close to the memory limit on 12 GB.
Strict on purpose. The date and currency rules leave null where a person might infer from context, such as a US street address without "USA".
Text only, English only. Scanned PDFs need OCR first.
Tax-inclusive and multi-currency invoices can produce a subtotal from the wrong figure.
Untrusted input. The model is trained to ignore instructions inside documents, but that is not a security boundary. Always run the validation in snippet 3.
Benchmark caveat. The benchmark comes from InvoiceParser Pro, a vendor, not an independent body. It is small and partly synthetic.
Not for unattended financial, tax or legal decisions. Keep a person in the loop.

Intended use: research and prototyping of private, on-device invoice extraction where every value must be traceable, feeding a deterministic validator and human review.

📁 Files

Table with columns: File, Purpose
File	Purpose
`adapter_model.safetensors`	LoRA weights, 132 MB, sha256 `e354a351…850ce83a`
`adapter_config.json`	PEFT config, base pinned to revision `7744afa`
`system_prompt.txt`	The exact system prompt used in training and evaluation
`tokenizer.json` · `tokenizer_config.json` · `chat_template.jinja`	Tokenizer and chat template
`invoice-v0.1.schema.json`	Output JSON Schema
`eval/release_metrics_v2.json`	v2 metrics, statuses and every field-level failure
`eval/release_metrics_v1_and_base.json`	v1 and prompted-base metrics
`assets/`	Charts on this card

📜 License and attribution

Adapter: Apache-2.0.
Base model: Qwen3-4B-Instruct-2507, Qwen team, Apache-2.0. 4-bit checkpoint by Unsloth.
Teacher model for layout re-rendering: gpt-oss-20b, OpenAI, Apache-2.0.
Evaluation data: invoice-extraction-benchmark by InvoiceParser Pro, CC BY 4.0, used for evaluation only.

📝 Citation

bibtex

@misc{privoid_extract_v2_2026,  title        = {Privoid Extract v2: Evidence-Grounded Invoice Extraction with a Qwen3-4B LoRA},  year         = {2026},  howpublished = {\url{https://huggingface.co/Privoid-ai/privoid-extract-qwen3-4b-lora}},  note         = {LoRA adapter for Qwen3-4B-Instruct-2507}}

@misc{privoid_extract_v2_2026,  title        = {Privoid Extract v2: Evidence-Grounded Invoice Extraction with a Qwen3-4B LoRA},  year         = {2026},  howpublished = {\url{https://huggingface.co/Privoid-ai/privoid-extract-qwen3-4b-lora}},  note         = {LoRA adapter for Qwen3-4B-Instruct-2507}}

privoid-extract-qwen3-4b-lora

README

🧾 What it returns

Field types

When it normalizes, and when it refuses to guess

🚀 Quickstart

📊 Benchmark

By field

By document category

Non-invoices

Held-out synthetic test set (60 documents)

Where v2 still misses

🏋️ Training

⚠️ Limitations

📁 Files

📜 License and attribution

📝 Citation

Explore FriendliAI today

README

🧾 What it returns

Field types

When it normalizes, and when it refuses to guess

🚀 Quickstart

📊 Benchmark

By field

By document category

Non-invoices

Held-out synthetic test set (60 documents)

Where v2 still misses

🏋️ Training

⚠️ Limitations

📁 Files

📜 License and attribution

📝 Citation