metadata
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd3
results: []
layoutlm-funsd3
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 0.6751
- Answer: {'precision': 0.6893617021276596, 'recall': 0.8009888751545118, 'f1': 0.7409948542024015, 'number': 809}
- Header: {'precision': 0.29, 'recall': 0.24369747899159663, 'f1': 0.2648401826484018, 'number': 119}
- Question: {'precision': 0.7557840616966581, 'recall': 0.828169014084507, 'f1': 0.7903225806451614, 'number': 1065}
- Overall Precision: 0.7064
- Overall Recall: 0.7822
- Overall F1: 0.7424
- Overall Accuracy: 0.8022
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 20
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 2.0338 | 1.0 | 10 | 2.0357 | {'precision': 0.037227214377406934, 'recall': 0.03584672435105068, 'f1': 0.03652392947103275, 'number': 809} | {'precision': 0.004568527918781726, 'recall': 0.15126050420168066, 'f1': 0.008869179600886918, 'number': 119} | {'precision': 0.052884615384615384, 'recall': 0.15492957746478872, 'f1': 0.07885304659498207, 'number': 1065} | 0.0270 | 0.1064 | 0.0431 | 0.0892 |
| 2.0223 | 2.0 | 20 | 2.0181 | {'precision': 0.03938730853391685, 'recall': 0.04449938195302843, 'f1': 0.04178757980266977, 'number': 809} | {'precision': 0.004824063564131668, 'recall': 0.14285714285714285, 'f1': 0.009332967334614329, 'number': 119} | {'precision': 0.05484848484848485, 'recall': 0.1699530516431925, 'f1': 0.08293241695303552, 'number': 1065} | 0.0302 | 0.1174 | 0.0481 | 0.1005 |
| 1.9986 | 3.0 | 30 | 1.9858 | {'precision': 0.04096989966555184, 'recall': 0.06056860321384425, 'f1': 0.048877805486284294, 'number': 809} | {'precision': 0.004677941705649514, 'recall': 0.1092436974789916, 'f1': 0.008971704623878536, 'number': 119} | {'precision': 0.05835468260745801, 'recall': 0.19248826291079812, 'f1': 0.08955875928352992, 'number': 1065} | 0.0357 | 0.1340 | 0.0563 | 0.1256 |
| 1.9605 | 4.0 | 40 | 1.9419 | {'precision': 0.03710462287104623, 'recall': 0.0754017305315204, 'f1': 0.04973501834488382, 'number': 809} | {'precision': 0.006282124500285551, 'recall': 0.09243697478991597, 'f1': 0.011764705882352943, 'number': 119} | {'precision': 0.06151288445552785, 'recall': 0.2084507042253521, 'f1': 0.09499358151476253, 'number': 1065} | 0.0420 | 0.1475 | 0.0654 | 0.1633 |
| 1.9119 | 5.0 | 50 | 1.8881 | {'precision': 0.03694581280788178, 'recall': 0.09270704573547589, 'f1': 0.0528355054596689, 'number': 809} | {'precision': 0.002574002574002574, 'recall': 0.01680672268907563, 'f1': 0.004464285714285714, 'number': 119} | {'precision': 0.0671203216826477, 'recall': 0.20375586854460093, 'f1': 0.10097719869706841, 'number': 1065} | 0.0487 | 0.1475 | 0.0732 | 0.2224 |
| 1.8502 | 6.0 | 60 | 1.8264 | {'precision': 0.03467062902426944, 'recall': 0.0865265760197775, 'f1': 0.04950495049504951, 'number': 809} | {'precision': 0.010101010101010102, 'recall': 0.01680672268907563, 'f1': 0.012618296529968456, 'number': 119} | {'precision': 0.08910070451719851, 'recall': 0.20187793427230047, 'f1': 0.1236342725704428, 'number': 1065} | 0.0620 | 0.1440 | 0.0867 | 0.2866 |
| 1.7869 | 7.0 | 70 | 1.7587 | {'precision': 0.026297085998578537, 'recall': 0.04573547589616811, 'f1': 0.033393501805054154, 'number': 809} | {'precision': 0.05555555555555555, 'recall': 0.008403361344537815, 'f1': 0.014598540145985401, 'number': 119} | {'precision': 0.13085399449035812, 'recall': 0.1784037558685446, 'f1': 0.15097338100913787, 'number': 1065} | 0.0792 | 0.1144 | 0.0936 | 0.3296 |
| 1.7064 | 8.0 | 80 | 1.6779 | {'precision': 0.018147086914995225, 'recall': 0.023485784919653894, 'f1': 0.020474137931034486, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.18514946962391515, 'recall': 0.18028169014084508, 'f1': 0.18268315889628925, 'number': 1065} | 0.1011 | 0.1059 | 0.1034 | 0.3535 |
| 1.6169 | 9.0 | 90 | 1.5857 | {'precision': 0.03464419475655431, 'recall': 0.04573547589616811, 'f1': 0.03942461374533831, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.2270450751252087, 'recall': 0.25539906103286386, 'f1': 0.2403888643393725, 'number': 1065} | 0.1364 | 0.1550 | 0.1451 | 0.3995 |
| 1.5331 | 10.0 | 100 | 1.4740 | {'precision': 0.06180344478216818, 'recall': 0.0754017305315204, 'f1': 0.06792873051224943, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.3259207783182766, 'recall': 0.4403755868544601, 'f1': 0.3746006389776358, 'number': 1065} | 0.2185 | 0.2659 | 0.2399 | 0.4731 |
| 1.3817 | 11.0 | 110 | 1.3317 | {'precision': 0.1468609865470852, 'recall': 0.1619283065512979, 'f1': 0.1540270429159318, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.42608089260808923, 'recall': 0.5737089201877934, 'f1': 0.48899559823929567, 'number': 1065} | 0.3190 | 0.3723 | 0.3436 | 0.5459 |
| 1.2192 | 12.0 | 120 | 1.1630 | {'precision': 0.2839506172839506, 'recall': 0.2843016069221261, 'f1': 0.28412600370599134, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5201238390092879, 'recall': 0.6309859154929578, 'f1': 0.5702163767501062, 'number': 1065} | 0.4283 | 0.4526 | 0.4401 | 0.6106 |
| 1.068 | 13.0 | 130 | 1.0176 | {'precision': 0.41460905349794236, 'recall': 0.49814585908529047, 'f1': 0.45255474452554745, 'number': 809} | {'precision': 0.05263157894736842, 'recall': 0.008403361344537815, 'f1': 0.014492753623188406, 'number': 119} | {'precision': 0.54477050413845, 'recall': 0.67981220657277, 'f1': 0.6048454469507102, 'number': 1065} | 0.4862 | 0.5660 | 0.5231 | 0.6807 |
| 0.901 | 14.0 | 140 | 0.9007 | {'precision': 0.484472049689441, 'recall': 0.5784919653893696, 'f1': 0.5273239436619719, 'number': 809} | {'precision': 0.023255813953488372, 'recall': 0.008403361344537815, 'f1': 0.01234567901234568, 'number': 119} | {'precision': 0.6263463131731566, 'recall': 0.7098591549295775, 'f1': 0.665492957746479, 'number': 1065} | 0.5528 | 0.6147 | 0.5821 | 0.7168 |
| 0.7884 | 15.0 | 150 | 0.8050 | {'precision': 0.5395833333333333, 'recall': 0.6402966625463535, 'f1': 0.5856416054267948, 'number': 809} | {'precision': 0.11940298507462686, 'recall': 0.06722689075630252, 'f1': 0.08602150537634408, 'number': 119} | {'precision': 0.6304176516942475, 'recall': 0.7511737089201878, 'f1': 0.6855184233076264, 'number': 1065} | 0.5775 | 0.6653 | 0.6183 | 0.7537 |
| 0.7027 | 16.0 | 160 | 0.7470 | {'precision': 0.6069246435845214, 'recall': 0.7367119901112484, 'f1': 0.6655499720826354, 'number': 809} | {'precision': 0.2236842105263158, 'recall': 0.14285714285714285, 'f1': 0.17435897435897438, 'number': 119} | {'precision': 0.6496465043205027, 'recall': 0.7765258215962442, 'f1': 0.7074422583404619, 'number': 1065} | 0.6178 | 0.7225 | 0.6660 | 0.7717 |
| 0.6177 | 17.0 | 170 | 0.7266 | {'precision': 0.6294691224268689, 'recall': 0.7181705809641533, 'f1': 0.6709006928406466, 'number': 809} | {'precision': 0.2777777777777778, 'recall': 0.16806722689075632, 'f1': 0.20942408376963353, 'number': 119} | {'precision': 0.6956875508543532, 'recall': 0.8028169014084507, 'f1': 0.7454228421970358, 'number': 1065} | 0.6547 | 0.7306 | 0.6905 | 0.7750 |
| 0.5539 | 18.0 | 180 | 0.6824 | {'precision': 0.6402805611222445, 'recall': 0.7898640296662547, 'f1': 0.7072495849474266, 'number': 809} | {'precision': 0.27710843373493976, 'recall': 0.19327731092436976, 'f1': 0.22772277227722776, 'number': 119} | {'precision': 0.7141687141687142, 'recall': 0.8187793427230047, 'f1': 0.762904636920385, 'number': 1065} | 0.6664 | 0.7697 | 0.7143 | 0.7913 |
| 0.499 | 19.0 | 190 | 0.6764 | {'precision': 0.6718266253869969, 'recall': 0.8046971569839307, 'f1': 0.732283464566929, 'number': 809} | {'precision': 0.3068181818181818, 'recall': 0.226890756302521, 'f1': 0.2608695652173913, 'number': 119} | {'precision': 0.7504378283712785, 'recall': 0.8046948356807512, 'f1': 0.7766198459447213, 'number': 1065} | 0.6980 | 0.7702 | 0.7323 | 0.7961 |
| 0.4355 | 20.0 | 200 | 0.6751 | {'precision': 0.6893617021276596, 'recall': 0.8009888751545118, 'f1': 0.7409948542024015, 'number': 809} | {'precision': 0.29, 'recall': 0.24369747899159663, 'f1': 0.2648401826484018, 'number': 119} | {'precision': 0.7557840616966581, 'recall': 0.828169014084507, 'f1': 0.7903225806451614, 'number': 1065} | 0.7064 | 0.7822 | 0.7424 | 0.8022 |
Framework versions
- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1