File size: 7,881 Bytes
b0213fe |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
---
license: mit
tags:
- generated_from_trainer
datasets:
- funsd-layoutlmv3
model-index:
- name: lilt-en-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# lilt-en-funsd
This model is a fine-tuned version of [SCUT-DLVCLab/lilt-roberta-en-base](https://huggingface.co/SCUT-DLVCLab/lilt-roberta-en-base) on the funsd-layoutlmv3 dataset.
It achieves the following results on the evaluation set:
- Loss: 1.8731
- Answer: {'precision': 0.8688915375446961, 'recall': 0.8922888616891065, 'f1': 0.8804347826086957, 'number': 817}
- Header: {'precision': 0.638095238095238, 'recall': 0.5630252100840336, 'f1': 0.5982142857142857, 'number': 119}
- Question: {'precision': 0.9105166051660517, 'recall': 0.9164345403899722, 'f1': 0.9134659879685332, 'number': 1077}
- Overall Precision: 0.8792
- Overall Recall: 0.8857
- Overall F1: 0.8825
- Overall Accuracy: 0.7976
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 2500
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:------:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 0.4323 | 10.53 | 200 | 1.0423 | {'precision': 0.8369195922989807, 'recall': 0.9045287637698899, 'f1': 0.8694117647058823, 'number': 817} | {'precision': 0.5405405405405406, 'recall': 0.5042016806722689, 'f1': 0.5217391304347826, 'number': 119} | {'precision': 0.8869323447636701, 'recall': 0.8885793871866295, 'f1': 0.8877551020408162, 'number': 1077} | 0.8471 | 0.8723 | 0.8595 | 0.7981 |
| 0.045 | 21.05 | 400 | 1.2757 | {'precision': 0.8435374149659864, 'recall': 0.9106487148102815, 'f1': 0.8758092995879929, 'number': 817} | {'precision': 0.5795454545454546, 'recall': 0.42857142857142855, 'f1': 0.49275362318840576, 'number': 119} | {'precision': 0.8626943005181347, 'recall': 0.9275766016713092, 'f1': 0.8939597315436242, 'number': 1077} | 0.8430 | 0.8912 | 0.8665 | 0.8026 |
| 0.0133 | 31.58 | 600 | 1.4887 | {'precision': 0.8632075471698113, 'recall': 0.8959608323133414, 'f1': 0.8792792792792793, 'number': 817} | {'precision': 0.6020408163265306, 'recall': 0.4957983193277311, 'f1': 0.543778801843318, 'number': 119} | {'precision': 0.8791887125220459, 'recall': 0.9257195914577531, 'f1': 0.9018543645409318, 'number': 1077} | 0.8596 | 0.8882 | 0.8737 | 0.7983 |
| 0.0051 | 42.11 | 800 | 1.7382 | {'precision': 0.8601645123384254, 'recall': 0.8959608323133414, 'f1': 0.8776978417266187, 'number': 817} | {'precision': 0.5636363636363636, 'recall': 0.5210084033613446, 'f1': 0.5414847161572053, 'number': 119} | {'precision': 0.9032558139534884, 'recall': 0.9015784586815228, 'f1': 0.9024163568773235, 'number': 1077} | 0.8669 | 0.8768 | 0.8718 | 0.7925 |
| 0.004 | 52.63 | 1000 | 1.7599 | {'precision': 0.8307349665924276, 'recall': 0.9130966952264382, 'f1': 0.8699708454810495, 'number': 817} | {'precision': 0.6039603960396039, 'recall': 0.5126050420168067, 'f1': 0.5545454545454545, 'number': 119} | {'precision': 0.8939256572982774, 'recall': 0.9155060352831941, 'f1': 0.9045871559633027, 'number': 1077} | 0.8530 | 0.8907 | 0.8714 | 0.7941 |
| 0.002 | 63.16 | 1200 | 1.8409 | {'precision': 0.8312985571587126, 'recall': 0.9167686658506732, 'f1': 0.8719441210710128, 'number': 817} | {'precision': 0.6074766355140186, 'recall': 0.5462184873949579, 'f1': 0.575221238938053, 'number': 119} | {'precision': 0.8814949863263446, 'recall': 0.8978644382544104, 'f1': 0.8896044158233671, 'number': 1077} | 0.8461 | 0.8847 | 0.8650 | 0.7876 |
| 0.0013 | 73.68 | 1400 | 1.7795 | {'precision': 0.81445523193096, 'recall': 0.9241126070991432, 'f1': 0.8658256880733943, 'number': 817} | {'precision': 0.6237623762376238, 'recall': 0.5294117647058824, 'f1': 0.5727272727272728, 'number': 119} | {'precision': 0.888785046728972, 'recall': 0.883008356545961, 'f1': 0.8858872845831393, 'number': 1077} | 0.8432 | 0.8788 | 0.8606 | 0.7934 |
| 0.0011 | 84.21 | 1600 | 1.8386 | {'precision': 0.8338833883388339, 'recall': 0.9277845777233782, 'f1': 0.8783314020857474, 'number': 817} | {'precision': 0.6597938144329897, 'recall': 0.5378151260504201, 'f1': 0.5925925925925926, 'number': 119} | {'precision': 0.8943985307621671, 'recall': 0.904363974001857, 'f1': 0.8993536472760849, 'number': 1077} | 0.8573 | 0.8922 | 0.8744 | 0.7945 |
| 0.0048 | 94.74 | 1800 | 1.8664 | {'precision': 0.8589595375722543, 'recall': 0.9094247246022031, 'f1': 0.8834720570749108, 'number': 817} | {'precision': 0.6504854368932039, 'recall': 0.5630252100840336, 'f1': 0.6036036036036037, 'number': 119} | {'precision': 0.9003656307129799, 'recall': 0.914577530176416, 'f1': 0.9074159373560571, 'number': 1077} | 0.8705 | 0.8917 | 0.8810 | 0.7927 |
| 0.0004 | 105.26 | 2000 | 1.8672 | {'precision': 0.8634772462077013, 'recall': 0.9057527539779682, 'f1': 0.8841099163679809, 'number': 817} | {'precision': 0.7093023255813954, 'recall': 0.5126050420168067, 'f1': 0.5951219512195123, 'number': 119} | {'precision': 0.8923076923076924, 'recall': 0.9155060352831941, 'f1': 0.9037580201649862, 'number': 1077} | 0.8726 | 0.8877 | 0.8801 | 0.7953 |
| 0.0005 | 115.79 | 2200 | 1.8731 | {'precision': 0.8688915375446961, 'recall': 0.8922888616891065, 'f1': 0.8804347826086957, 'number': 817} | {'precision': 0.638095238095238, 'recall': 0.5630252100840336, 'f1': 0.5982142857142857, 'number': 119} | {'precision': 0.9105166051660517, 'recall': 0.9164345403899722, 'f1': 0.9134659879685332, 'number': 1077} | 0.8792 | 0.8857 | 0.8825 | 0.7976 |
| 0.0002 | 126.32 | 2400 | 1.9408 | {'precision': 0.8408071748878924, 'recall': 0.9179926560587516, 'f1': 0.8777062609713283, 'number': 817} | {'precision': 0.6310679611650486, 'recall': 0.5462184873949579, 'f1': 0.5855855855855856, 'number': 119} | {'precision': 0.9091760299625468, 'recall': 0.9015784586815228, 'f1': 0.9053613053613054, 'number': 1077} | 0.8657 | 0.8872 | 0.8763 | 0.7935 |
### Framework versions
- Transformers 4.24.0
- Pytorch 1.12.1+cu113
- Datasets 2.7.1
- Tokenizers 0.13.2
|