|
--- |
|
license: cdla-permissive-2.0 |
|
--- |
|
|
|
# Docling Models |
|
|
|
This page contains models that power the PDF document converion package [docling](https://github.com/DS4SD/docling). |
|
|
|
## Layout Model |
|
|
|
The layout model will take an image from a poge and apply RT-DETR model in order to find different layout components. It currently detects the labels: Caption, Footnote, Formula, List-item, Page-footer, Page-header, Picture, Section-header, Table, Text, Title. As a reference (from the DocLayNet-paper), this is the performance of standard object detection methods on the DocLayNet dataset compared to human evaluation, |
|
|
|
| | human | MRCNN | MRCNN | FRCNN | YOLO | |
|
|----------------|---------|---------|---------|---------|--------| |
|
| | human | R50 | R101 | R101 | v5x6 | |
|
| Caption | 84-89 | 68.4 | 71.5 | 70.1 | 77.7 | |
|
| Footnote | 83-91 | 70.9 | 71.8 | 73.7 | 77.2 | |
|
| Formula | 83-85 | 60.1 | 63.4 | 63.5 | 66.2 | |
|
| List-item | 87-88 | 81.2 | 80.8 | 81.0 | 86.2 | |
|
| Page-footer | 93-94 | 61.6 | 59.3 | 58.9 | 61.1 | |
|
| Page-header | 85-89 | 71.9 | 70.0 | 72.0 | 67.9 | |
|
| Picture | 69-71 | 71.7 | 72.7 | 72.0 | 77.1 | |
|
| Section-header | 83-84 | 67.6 | 69.3 | 68.4 | 74.6 | |
|
| Table | 77-81 | 82.2 | 82.9 | 82.2 | 86.3 | |
|
| Text | 84-86 | 84.6 | 85.8 | 85.4 | 88.1 | |
|
| Title | 60-72 | 76.7 | 80.4 | 79.9 | 82.7 | |
|
| All | 82-83 | 72.4 | 73.5 | 73.4 | 76.8 | |
|
|
|
## TableFormer |
|
|
|
The tableformer model will identify the structure of the table, starting from an image of a table. It uses the predicted table regions of the layout model to identify the tables. Tableformer has SOTA table structure identification, |
|
|
|
| Model (TEDS) | Simple table | Complex table | All tables | |
|
| ------------ | ------------ | ------------- | ---------- | |
|
| Tabula | 78.0 | 57.8 | 67.9 | |
|
| Traprange | 60.8 | 49.9 | 55.4 | |
|
| Camelot | 80.0 | 66.0 | 73.0 | |
|
| Acrobat Pro | 68.9 | 61.8 | 65.3 | |
|
| EDD | 91.2 | 85.4 | 88.3 | |
|
| TableFormer | 95.4 | 90.1 | 93.6 | |
|
|
|
## References |
|
|
|
``` |
|
@techreport{Docling, |
|
author = {Deep Search Team}, |
|
month = {8}, |
|
title = {{Docling Technical Report}}, |
|
url={https://arxiv.org/abs/2408.09869}, |
|
eprint={2408.09869}, |
|
doi = "10.48550/arXiv.2408.09869", |
|
version = {1.0.0}, |
|
year = {2024} |
|
} |
|
|
|
@article{doclaynet2022, |
|
title = {DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis}, |
|
doi = {10.1145/3534678.353904}, |
|
url = {https://arxiv.org/abs/2206.01062}, |
|
author = {Pfitzmann, Birgit and Auer, Christoph and Dolfi, Michele and Nassar, Ahmed S and Staar, Peter W J}, |
|
year = {2022} |
|
} |
|
|
|
@InProceedings{TableFormer2022, |
|
author = {Nassar, Ahmed and Livathinos, Nikolaos and Lysak, Maksym and Staar, Peter}, |
|
title = {TableFormer: Table Structure Understanding With Transformers}, |
|
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, |
|
month = {June}, |
|
year = {2022}, |
|
pages = {4614-4623}, |
|
doi = {https://doi.org/10.1109/CVPR52688.2022.00457} |
|
} |
|
``` |
|
|