What changed
LoRA (rank 16) on the decoder and the vision blocks, 2 epochs, then merged into the weights. So this repo is a full model, not an adapter.
The labels came from the base model itself. It reads a table well when the page is upright, so I took that upright reading as the label, rotated the image, and trained the model to give the same output. A label was kept only when its money values matched the PDF text layer exactly. No teacher model and no hand labels. The training set was 1,488 pages built from 134 rotated tables, all from public reports of Vietnamese listed companies.
Results
All numbers are on issuers that were not in training, same machine for both models, temperature 0, max 8192 tokens. "Recall" is the share of money values in the PDF text layer that appear in the output.
Table with columns: Test, Base, This model| Test | Base | This model |
|---|
| Two-page spread, upright page + rotated table (124 pages), recall | 90.8% | 99.4% |
| Two-page spread, rotated table + upright page (83 pages), recall | 92.6% | 99.1% |
| Two rotated tables in one spread (41 pages), recall | 92.5% | 99.1% |
| Rotated table alone, 1448 px, recall | 96.2% | 99.5% |
| Rotated table alone, 1152 px, recall | 90.9% | 94.1% |
| Two upright pages (control), recall | 98.4% | 100% |
| Real annual-report spreads with rotated text (42 pages), recall | 96.3% | 99.2% |
| 68 hard pages, recall / precision | 99.4% / 91.3% | 99.8% / 98.3% |
On general documents it is slightly worse than the base. OmniDocBench v1.6, all 1,651 pages, official evaluator without CDM, raw model output:
Table with columns: Metric, Base, This model| Metric | Base | This model |
|---|
| Text edit distance (lower is better) | 0.0821 | 0.0824 |
| Table TEDS | 79.52 | 78.75 |
| Reading order edit distance | 0.1733 | 0.1708 |
These absolute numbers come from my own run with raw output and may be lower than published ones. The comparison between the two columns is what counts.
On an English report with plain number tables (USACE port tonnage, 150 ports) both models get every value right.
Limits
- Tested on born-digital PDF pages. Real scans with rotated tables are not tested yet.
- The held-out test has few issuers: 5 for the recombined pages and 9 for the real spreads.
- A rotated table rendered small (1152 px on the long edge) is still weak at 94.1%.
- It loops on a few pages where the base does not (3 of 344 real spreads), and it removes 8 loops the base had. Catch pages that hit the token limit and read them again, as you would with the base.
- If your documents are Chinese books, slides or magazines, the base model is the better pick.
License
Apache 2.0, the same as the base model. All credit for the model itself goes to LightOn. See the base model card for the paper and citation.