---
license: gpl-3.0
tags:
- DocVQA
- Document Question Answering
- Document Visual Question Answering
datasets:
- rubentito/mp-docvqa
language:
- en
---

# BERT-LARGE fine-tuned on MP-DocVQA

This is BERT trained on [SinglePage DocVQA](https://arxiv.org/abs/2007.00398) and fine-tuned on Multipage DocVQA (MP-DocVQA) dataset.

This model was used as a baseline in [Hierarchical multimodal transformers for Multi-Page DocVQA](https://arxiv.org/pdf/2212.05935.pdf).
- Training hyperparameters can be found in Table 8 of Appendix D.

## How to use

Here is how to use this model to get the features of a given text in PyTorch:

```python
from transformers import AutoModelForQuestionAnswering, AutoTokenizer

model = AutoModelForQuestionAnswering.from_pretrained("rubentito/bert-large-mpdocvqa")

question = "Replace me by any text you'd like."
context = "Put some context for answering"
encoded_input = tokenizer(question, context, return_tensors='pt')
output = model(**encoded_input)
```

## Model results

Extended experimentation can be found in Table 2 of [Hierarchical multimodal transformers for Multi-Page DocVQA](https://arxiv.org/pdf/2212.05935.pdf).
You can also check the live leaderboard at the [RRC Portal](https://rrc.cvc.uab.es/?ch=17&com=evaluation&task=4).
| Model 		 																	| HF name								| ANLS 			| APPA		|
|-----------------------------------------------------------------------------------|:--------------------------------------|:-------------:|:---------:|
| [**Bert large**](https://huggingface.co/rubentito/bert-large-mpdocvqa)	        | rubentito/bert-large-mpdocvqa			| 0.4183 		| 51.6177 	|
| [Longformer base](https://huggingface.co/rubentito/longformer-base-mpdocvqa)		| rubentito/longformer-base-mpdocvqa	| 0.5287		| 71.1696 	|
| [BigBird ITC base](https://huggingface.co/rubentito/bigbird-base-itc-mpdocvqa)	| rubentito/bigbird-base-itc-mpdocvqa	| 0.4929		| 67.5433 	|
| [LayoutLMv3 base](https://huggingface.co/rubentito/layoutlmv3-base-mpdocvqa)		| rubentito/layoutlmv3-base-mpdocvqa	| 0.4538		| 51.9426 	|
| [T5 base](https://huggingface.co/rubentito/t5-base-mpdocvqa)						| rubentito/t5-base-mpdocvqa			| 0.5050		| 0.0000 	|
| Hi-VT5 																			| TBA 									| 0.6201		| 79.23		|

## Citation Information 

```tex
@article{tito2022hierarchical,
  title={Hierarchical multimodal transformers for Multi-Page DocVQA},
  author={Tito, Rub{\`e}n and Karatzas, Dimosthenis and Valveny, Ernest},
  journal={arXiv preprint arXiv:2212.05935},
  year={2022}
}
```