Results
Test sets, greedy decoding, fp16, adapter merged. Field F1 is the micro F1 over
(field path, normalized value) pairs; exact match requires every field to be right.
Brackets give 95% bootstrap intervals.
Table with columns: Test set, Model, Valid JSON, Exact match, Field F1| Test set | Model | Valid JSON | Exact match | Field F1 |
|---|
| CORD v2 test (100) | base model | 0.89 | 0.00 [0.00, 0.00] | 0.351 [0.322, 0.381] |
| this adapter | 0.99 | 0.35 [0.26, 0.45] | 0.832 [0.789, 0.873] |
| Synthetic invoices test (100) | base model | 0.99 | 0.13 [0.07, 0.21] | 0.891 [0.866, 0.912] |
| this adapter | 1.00 | 0.63 [0.53, 0.72] | 0.979 [0.972, 0.985] |
| of which unseen templates (50) | this adapter | 1.00 | 0.54 [0.40, 0.68] | 0.974 [0.963, 0.985] |
Latency on a T4 at batch size 1: 5.8 s per receipt and 14.3 s per invoice on average,
4.5 GB peak GPU memory.
Training
One epoch on the 800 CORD v2 training receipts and 1000 synthetic French invoices.
QLoRA: 4-bit NF4 base with double quantization, fp16, LoRA r=16, alpha 32, dropout 0.05 on the
q, k, v, o, gate, up and down projections of the language model (17.4M parameters).
Batch size 8 (2 x 4 accumulation), learning rate 2e-4 with cosine decay, 225 steps on one T4.
Usage
The prompts are part of the training setup; use the package from the code repository
(pip install -r app/requirements.txt && pip install -e . in a clone):
from PIL import Imagefrom receipt_vlm.service import extract result = extract(Image.open("invoice.jpg"), kind="invoice") # kind: "invoice" or "cord"print(result["data"])
Limits
- No real invoice was used for training or evaluation; results on real scans are unknown.
- The synthetic invoices come from 6 HTML templates; unseen templates only change the layout.
- Rare CORD fields (sub-items,
itemsubtotal, etc) are still often missed.
Licenses
CORD citation:
@inproceedings{park2019cord, title={CORD: A Consolidated Receipt Dataset for Post-OCR Parsing}, author={Park, Seunghyun and Shin, Seung and Lee, Bado and Lee, Junyeop and Surh, Jaeheung and Seo, Minjoon and Lee, Hwalsuk}, booktitle={Document Intelligence Workshop at Neural Information Processing Systems}, year={2019}}