File size: 9,277 Bytes
915bfe1 d21a17a 915bfe1 d21a17a 915bfe1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 |
---
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 1.0436
- Answer: {'precision': 0.3978685612788632, 'recall': 0.553770086526576, 'f1': 0.4630490956072351, 'number': 809}
- Header: {'precision': 0.32926829268292684, 'recall': 0.226890756302521, 'f1': 0.26865671641791045, 'number': 119}
- Question: {'precision': 0.5241157556270096, 'recall': 0.612206572769953, 'f1': 0.5647466435686443, 'number': 1065}
- Overall Precision: 0.4596
- Overall Recall: 0.5655
- Overall F1: 0.5071
- Overall Accuracy: 0.6267
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.7148 | 1.0 | 10 | 1.5016 | {'precision': 0.08819018404907976, 'recall': 0.14215080346106304, 'f1': 0.10884997633696165, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.2198581560283688, 'recall': 0.08732394366197183, 'f1': 0.125, 'number': 1065} | 0.1204 | 0.1044 | 0.1118 | 0.3613 |
| 1.4202 | 2.0 | 20 | 1.3572 | {'precision': 0.21160042964554243, 'recall': 0.48702101359703337, 'f1': 0.29502059153874954, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.24895977808599168, 'recall': 0.3370892018779343, 'f1': 0.28639808536098926, 'number': 1065} | 0.2265 | 0.3778 | 0.2832 | 0.4216 |
| 1.2863 | 3.0 | 30 | 1.2150 | {'precision': 0.25656167979002625, 'recall': 0.48331273176761436, 'f1': 0.33519074153450495, 'number': 809} | {'precision': 0.06779661016949153, 'recall': 0.03361344537815126, 'f1': 0.0449438202247191, 'number': 119} | {'precision': 0.3437908496732026, 'recall': 0.49389671361502346, 'f1': 0.4053949903660886, 'number': 1065} | 0.2959 | 0.4621 | 0.3608 | 0.4790 |
| 1.1633 | 4.0 | 40 | 1.1144 | {'precision': 0.2625454545454545, 'recall': 0.446229913473424, 'f1': 0.3305860805860806, 'number': 809} | {'precision': 0.3253012048192771, 'recall': 0.226890756302521, 'f1': 0.26732673267326734, 'number': 119} | {'precision': 0.37986577181208053, 'recall': 0.5314553990610329, 'f1': 0.4430528375733855, 'number': 1065} | 0.3236 | 0.4787 | 0.3862 | 0.5442 |
| 1.0585 | 5.0 | 50 | 1.0827 | {'precision': 0.3039940828402367, 'recall': 0.5080346106304079, 'f1': 0.38037945395650163, 'number': 809} | {'precision': 0.32432432432432434, 'recall': 0.20168067226890757, 'f1': 0.24870466321243526, 'number': 119} | {'precision': 0.4149933065595716, 'recall': 0.5821596244131455, 'f1': 0.48456428292301673, 'number': 1065} | 0.3613 | 0.5294 | 0.4295 | 0.5700 |
| 0.9987 | 6.0 | 60 | 1.0373 | {'precision': 0.326783114992722, 'recall': 0.5550061804697157, 'f1': 0.4113605130554283, 'number': 809} | {'precision': 0.4074074074074074, 'recall': 0.18487394957983194, 'f1': 0.2543352601156069, 'number': 119} | {'precision': 0.453125, 'recall': 0.5173708920187794, 'f1': 0.4831214379658045, 'number': 1065} | 0.3865 | 0.5128 | 0.4408 | 0.6016 |
| 0.9315 | 7.0 | 70 | 1.0055 | {'precision': 0.34718100890207715, 'recall': 0.4338689740420272, 'f1': 0.3857142857142857, 'number': 809} | {'precision': 0.3229166666666667, 'recall': 0.2605042016806723, 'f1': 0.28837209302325584, 'number': 119} | {'precision': 0.4558011049723757, 'recall': 0.6197183098591549, 'f1': 0.5252686032630322, 'number': 1065} | 0.4078 | 0.5228 | 0.4582 | 0.6164 |
| 0.8716 | 8.0 | 80 | 1.0112 | {'precision': 0.33733013589128696, 'recall': 0.5216316440049443, 'f1': 0.40970873786407763, 'number': 809} | {'precision': 0.3717948717948718, 'recall': 0.24369747899159663, 'f1': 0.29441624365482233, 'number': 119} | {'precision': 0.44542372881355935, 'recall': 0.6169014084507042, 'f1': 0.5173228346456693, 'number': 1065} | 0.3951 | 0.5559 | 0.4620 | 0.6153 |
| 0.8102 | 9.0 | 90 | 1.0152 | {'precision': 0.3773062730627306, 'recall': 0.5055624227441285, 'f1': 0.4321183306920232, 'number': 809} | {'precision': 0.3611111111111111, 'recall': 0.2184873949579832, 'f1': 0.27225130890052357, 'number': 119} | {'precision': 0.4880860876249039, 'recall': 0.596244131455399, 'f1': 0.536770921386306, 'number': 1065} | 0.4355 | 0.5369 | 0.4809 | 0.6226 |
| 0.8003 | 10.0 | 100 | 1.0342 | {'precision': 0.3804878048780488, 'recall': 0.5784919653893696, 'f1': 0.45904855321235905, 'number': 809} | {'precision': 0.32, 'recall': 0.20168067226890757, 'f1': 0.24742268041237112, 'number': 119} | {'precision': 0.5183887915936952, 'recall': 0.5558685446009389, 'f1': 0.5364748527412777, 'number': 1065} | 0.4430 | 0.5439 | 0.4883 | 0.6143 |
| 0.728 | 11.0 | 110 | 1.0330 | {'precision': 0.3871559633027523, 'recall': 0.5216316440049443, 'f1': 0.4444444444444445, 'number': 809} | {'precision': 0.29213483146067415, 'recall': 0.2184873949579832, 'f1': 0.25, 'number': 119} | {'precision': 0.4981791697013838, 'recall': 0.6422535211267606, 'f1': 0.561115668580804, 'number': 1065} | 0.4436 | 0.5680 | 0.4981 | 0.6221 |
| 0.7175 | 12.0 | 120 | 1.0841 | {'precision': 0.38127090301003347, 'recall': 0.5636588380716935, 'f1': 0.45486284289276807, 'number': 809} | {'precision': 0.3684210526315789, 'recall': 0.23529411764705882, 'f1': 0.28717948717948716, 'number': 119} | {'precision': 0.5153225806451613, 'recall': 0.6, 'f1': 0.5544468546637744, 'number': 1065} | 0.4471 | 0.5635 | 0.4986 | 0.6243 |
| 0.6893 | 13.0 | 130 | 1.0501 | {'precision': 0.3815126050420168, 'recall': 0.5611866501854141, 'f1': 0.4542271135567784, 'number': 809} | {'precision': 0.30952380952380953, 'recall': 0.2184873949579832, 'f1': 0.2561576354679803, 'number': 119} | {'precision': 0.5256950294860994, 'recall': 0.5859154929577465, 'f1': 0.5541740674955595, 'number': 1065} | 0.4486 | 0.5539 | 0.4957 | 0.6228 |
| 0.653 | 14.0 | 140 | 1.0222 | {'precision': 0.39345794392523364, 'recall': 0.5203955500618047, 'f1': 0.4481106971793507, 'number': 809} | {'precision': 0.34615384615384615, 'recall': 0.226890756302521, 'f1': 0.27411167512690354, 'number': 119} | {'precision': 0.5045180722891566, 'recall': 0.6291079812206573, 'f1': 0.55996656916005, 'number': 1065} | 0.4515 | 0.5610 | 0.5003 | 0.6269 |
| 0.6494 | 15.0 | 150 | 1.0436 | {'precision': 0.3978685612788632, 'recall': 0.553770086526576, 'f1': 0.4630490956072351, 'number': 809} | {'precision': 0.32926829268292684, 'recall': 0.226890756302521, 'f1': 0.26865671641791045, 'number': 119} | {'precision': 0.5241157556270096, 'recall': 0.612206572769953, 'f1': 0.5647466435686443, 'number': 1065} | 0.4596 | 0.5655 | 0.5071 | 0.6267 |
### Framework versions
- Transformers 4.38.2
- Pytorch 2.1.0+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
|