File size: 9,335 Bytes
90076d7 4dfa76b 90076d7 4dfa76b 90076d7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6806
- Answer: {'precision': 0.709211986681465, 'recall': 0.7898640296662547, 'f1': 0.7473684210526316, 'number': 809}
- Header: {'precision': 0.35537190082644626, 'recall': 0.36134453781512604, 'f1': 0.3583333333333333, 'number': 119}
- Question: {'precision': 0.7920792079207921, 'recall': 0.8262910798122066, 'f1': 0.8088235294117647, 'number': 1065}
- Overall Precision: 0.7323
- Overall Recall: 0.7837
- Overall F1: 0.7571
- Overall Accuracy: 0.8125
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.7526 | 1.0 | 10 | 1.5590 | {'precision': 0.032426778242677826, 'recall': 0.038318912237330034, 'f1': 0.03512747875354107, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.23852295409181637, 'recall': 0.2244131455399061, 'f1': 0.2312530237058539, 'number': 1065} | 0.1379 | 0.1355 | 0.1367 | 0.3812 |
| 1.4179 | 2.0 | 20 | 1.2477 | {'precision': 0.16770186335403728, 'recall': 0.1668726823238566, 'f1': 0.16728624535315983, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4325309992706054, 'recall': 0.5568075117370892, 'f1': 0.486863711001642, 'number': 1065} | 0.3343 | 0.3653 | 0.3491 | 0.5813 |
| 1.0864 | 3.0 | 30 | 0.9440 | {'precision': 0.5470383275261324, 'recall': 0.5822002472187886, 'f1': 0.5640718562874251, 'number': 809} | {'precision': 0.0425531914893617, 'recall': 0.01680672268907563, 'f1': 0.024096385542168672, 'number': 119} | {'precision': 0.5717665615141956, 'recall': 0.6807511737089202, 'f1': 0.6215173596228033, 'number': 1065} | 0.5506 | 0.6011 | 0.5747 | 0.7225 |
| 0.8353 | 4.0 | 40 | 0.7733 | {'precision': 0.5964360587002097, 'recall': 0.7033374536464772, 'f1': 0.6454906409529211, 'number': 809} | {'precision': 0.19718309859154928, 'recall': 0.11764705882352941, 'f1': 0.14736842105263157, 'number': 119} | {'precision': 0.654468085106383, 'recall': 0.7220657276995305, 'f1': 0.6866071428571429, 'number': 1065} | 0.6145 | 0.6784 | 0.6449 | 0.7634 |
| 0.6716 | 5.0 | 50 | 0.7154 | {'precision': 0.6294691224268689, 'recall': 0.7181705809641533, 'f1': 0.6709006928406466, 'number': 809} | {'precision': 0.24210526315789474, 'recall': 0.19327731092436976, 'f1': 0.2149532710280374, 'number': 119} | {'precision': 0.6755663430420712, 'recall': 0.784037558685446, 'f1': 0.7257714037375055, 'number': 1065} | 0.6384 | 0.7220 | 0.6777 | 0.7796 |
| 0.5748 | 6.0 | 60 | 0.6924 | {'precision': 0.6378269617706237, 'recall': 0.7836835599505563, 'f1': 0.7032723239046034, 'number': 809} | {'precision': 0.3493975903614458, 'recall': 0.24369747899159663, 'f1': 0.2871287128712871, 'number': 119} | {'precision': 0.7334558823529411, 'recall': 0.7492957746478873, 'f1': 0.7412912215513237, 'number': 1065} | 0.6748 | 0.7331 | 0.7027 | 0.7798 |
| 0.5 | 7.0 | 70 | 0.6652 | {'precision': 0.665258711721225, 'recall': 0.7787391841779975, 'f1': 0.7175398633257404, 'number': 809} | {'precision': 0.2641509433962264, 'recall': 0.23529411764705882, 'f1': 0.24888888888888888, 'number': 119} | {'precision': 0.7253218884120172, 'recall': 0.7934272300469484, 'f1': 0.7578475336322871, 'number': 1065} | 0.6776 | 0.7541 | 0.7138 | 0.7942 |
| 0.4449 | 8.0 | 80 | 0.6592 | {'precision': 0.6754201680672269, 'recall': 0.7948084054388134, 'f1': 0.730266893810335, 'number': 809} | {'precision': 0.25862068965517243, 'recall': 0.25210084033613445, 'f1': 0.25531914893617025, 'number': 119} | {'precision': 0.7574692442882249, 'recall': 0.8093896713615023, 'f1': 0.7825692237857468, 'number': 1065} | 0.6958 | 0.7702 | 0.7311 | 0.8050 |
| 0.3916 | 9.0 | 90 | 0.6470 | {'precision': 0.7090301003344481, 'recall': 0.7861557478368356, 'f1': 0.7456037514654162, 'number': 809} | {'precision': 0.3157894736842105, 'recall': 0.3025210084033613, 'f1': 0.30901287553648066, 'number': 119} | {'precision': 0.762071992976295, 'recall': 0.8150234741784037, 'f1': 0.7876588021778583, 'number': 1065} | 0.7163 | 0.7727 | 0.7434 | 0.8102 |
| 0.3807 | 10.0 | 100 | 0.6552 | {'precision': 0.6869009584664537, 'recall': 0.7972805933250927, 'f1': 0.7379862700228833, 'number': 809} | {'precision': 0.2972972972972973, 'recall': 0.2773109243697479, 'f1': 0.28695652173913044, 'number': 119} | {'precision': 0.7832422586520947, 'recall': 0.8075117370892019, 'f1': 0.7951918631530283, 'number': 1065} | 0.7160 | 0.7717 | 0.7428 | 0.8129 |
| 0.328 | 11.0 | 110 | 0.6710 | {'precision': 0.7014428412874584, 'recall': 0.7812113720642769, 'f1': 0.7391812865497076, 'number': 809} | {'precision': 0.3037037037037037, 'recall': 0.3445378151260504, 'f1': 0.3228346456692913, 'number': 119} | {'precision': 0.7671589921807124, 'recall': 0.8291079812206573, 'f1': 0.7969314079422383, 'number': 1065} | 0.7115 | 0.7807 | 0.7445 | 0.8076 |
| 0.3111 | 12.0 | 120 | 0.6772 | {'precision': 0.6972972972972973, 'recall': 0.7972805933250927, 'f1': 0.7439446366782007, 'number': 809} | {'precision': 0.34234234234234234, 'recall': 0.31932773109243695, 'f1': 0.33043478260869563, 'number': 119} | {'precision': 0.801477377654663, 'recall': 0.8150234741784037, 'f1': 0.8081936685288641, 'number': 1065} | 0.7319 | 0.7782 | 0.7544 | 0.8120 |
| 0.2936 | 13.0 | 130 | 0.6751 | {'precision': 0.7136563876651982, 'recall': 0.8009888751545118, 'f1': 0.7548048922539313, 'number': 809} | {'precision': 0.33858267716535434, 'recall': 0.36134453781512604, 'f1': 0.34959349593495936, 'number': 119} | {'precision': 0.7894736842105263, 'recall': 0.8309859154929577, 'f1': 0.8096980786825252, 'number': 1065} | 0.7310 | 0.7908 | 0.7597 | 0.8126 |
| 0.2719 | 14.0 | 140 | 0.6794 | {'precision': 0.7081021087680355, 'recall': 0.788627935723115, 'f1': 0.7461988304093568, 'number': 809} | {'precision': 0.3524590163934426, 'recall': 0.36134453781512604, 'f1': 0.35684647302904565, 'number': 119} | {'precision': 0.794755877034358, 'recall': 0.8253521126760563, 'f1': 0.809765085214187, 'number': 1065} | 0.7327 | 0.7827 | 0.7569 | 0.8116 |
| 0.2776 | 15.0 | 150 | 0.6806 | {'precision': 0.709211986681465, 'recall': 0.7898640296662547, 'f1': 0.7473684210526316, 'number': 809} | {'precision': 0.35537190082644626, 'recall': 0.36134453781512604, 'f1': 0.3583333333333333, 'number': 119} | {'precision': 0.7920792079207921, 'recall': 0.8262910798122066, 'f1': 0.8088235294117647, 'number': 1065} | 0.7323 | 0.7837 | 0.7571 | 0.8125 |
### Framework versions
- Transformers 4.41.1
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1
|