metadata
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
layoutlm-funsd
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 0.7055
- Answer: {'precision': 0.7035830618892508, 'recall': 0.8009888751545118, 'f1': 0.7491329479768787, 'number': 809}
- Header: {'precision': 0.34146341463414637, 'recall': 0.35294117647058826, 'f1': 0.34710743801652894, 'number': 119}
- Question: {'precision': 0.7775816416593115, 'recall': 0.8272300469483568, 'f1': 0.8016378525932666, 'number': 1065}
- Overall Precision: 0.7216
- Overall Recall: 0.7883
- Overall F1: 0.7535
- Overall Accuracy: 0.8028
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.8301 | 1.0 | 10 | 1.5849 | {'precision': 0.008086253369272238, 'recall': 0.007416563658838072, 'f1': 0.007736943907156674, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.22358346094946402, 'recall': 0.13708920187793427, 'f1': 0.16996507566938301, 'number': 1065} | 0.1090 | 0.0763 | 0.0897 | 0.3514 |
| 1.4704 | 2.0 | 20 | 1.2710 | {'precision': 0.2843881856540084, 'recall': 0.41656365883807167, 'f1': 0.3380140421263791, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.3906474820143885, 'recall': 0.5098591549295775, 'f1': 0.44236252545824845, 'number': 1065} | 0.3408 | 0.4415 | 0.3847 | 0.6020 |
| 1.1259 | 3.0 | 30 | 0.9451 | {'precision': 0.47373447946513847, 'recall': 0.6131025957972805, 'f1': 0.5344827586206896, 'number': 809} | {'precision': 0.0625, 'recall': 0.025210084033613446, 'f1': 0.035928143712574856, 'number': 119} | {'precision': 0.5223654283548143, 'recall': 0.6469483568075117, 'f1': 0.5780201342281879, 'number': 1065} | 0.4921 | 0.5961 | 0.5391 | 0.7000 |
| 0.8549 | 4.0 | 40 | 0.7891 | {'precision': 0.5652985074626866, 'recall': 0.7490729295426453, 'f1': 0.6443381180223287, 'number': 809} | {'precision': 0.20833333333333334, 'recall': 0.12605042016806722, 'f1': 0.15706806282722513, 'number': 119} | {'precision': 0.6485013623978202, 'recall': 0.6704225352112676, 'f1': 0.6592797783933518, 'number': 1065} | 0.5947 | 0.6698 | 0.6300 | 0.7562 |
| 0.6872 | 5.0 | 50 | 0.7203 | {'precision': 0.6393617021276595, 'recall': 0.7428924598269468, 'f1': 0.6872498570611778, 'number': 809} | {'precision': 0.358974358974359, 'recall': 0.23529411764705882, 'f1': 0.28426395939086296, 'number': 119} | {'precision': 0.6650563607085346, 'recall': 0.7755868544600939, 'f1': 0.716081491114001, 'number': 1065} | 0.6438 | 0.7301 | 0.6842 | 0.7798 |
| 0.5872 | 6.0 | 60 | 0.6889 | {'precision': 0.6236559139784946, 'recall': 0.788627935723115, 'f1': 0.6965065502183407, 'number': 809} | {'precision': 0.35802469135802467, 'recall': 0.24369747899159663, 'f1': 0.29000000000000004, 'number': 119} | {'precision': 0.7190517998244074, 'recall': 0.7690140845070422, 'f1': 0.7431941923774955, 'number': 1065} | 0.6625 | 0.7456 | 0.7016 | 0.7797 |
| 0.5065 | 7.0 | 70 | 0.6618 | {'precision': 0.681283422459893, 'recall': 0.7873918417799752, 'f1': 0.7305045871559632, 'number': 809} | {'precision': 0.336734693877551, 'recall': 0.2773109243697479, 'f1': 0.30414746543778803, 'number': 119} | {'precision': 0.748471615720524, 'recall': 0.8046948356807512, 'f1': 0.7755656108597285, 'number': 1065} | 0.7011 | 0.7662 | 0.7322 | 0.7934 |
| 0.4527 | 8.0 | 80 | 0.6639 | {'precision': 0.671161825726141, 'recall': 0.799752781211372, 'f1': 0.7298364354201917, 'number': 809} | {'precision': 0.3170731707317073, 'recall': 0.3277310924369748, 'f1': 0.32231404958677684, 'number': 119} | {'precision': 0.7473867595818815, 'recall': 0.8056338028169014, 'f1': 0.7754179846362403, 'number': 1065} | 0.6908 | 0.7747 | 0.7304 | 0.7955 |
| 0.3952 | 9.0 | 90 | 0.6666 | {'precision': 0.686358754027927, 'recall': 0.7898640296662547, 'f1': 0.7344827586206897, 'number': 809} | {'precision': 0.3523809523809524, 'recall': 0.31092436974789917, 'f1': 0.33035714285714285, 'number': 119} | {'precision': 0.7519247219846023, 'recall': 0.8253521126760563, 'f1': 0.7869292748433303, 'number': 1065} | 0.7052 | 0.7802 | 0.7408 | 0.7969 |
| 0.3863 | 10.0 | 100 | 0.6806 | {'precision': 0.6849894291754757, 'recall': 0.8009888751545118, 'f1': 0.7384615384615385, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.31932773109243695, 'f1': 0.3261802575107296, 'number': 119} | {'precision': 0.7670157068062827, 'recall': 0.8253521126760563, 'f1': 0.7951153324287653, 'number': 1065} | 0.7094 | 0.7852 | 0.7454 | 0.7985 |
| 0.3307 | 11.0 | 110 | 0.6859 | {'precision': 0.6938775510204082, 'recall': 0.7985166872682324, 'f1': 0.7425287356321839, 'number': 809} | {'precision': 0.3416666666666667, 'recall': 0.3445378151260504, 'f1': 0.34309623430962344, 'number': 119} | {'precision': 0.764402407566638, 'recall': 0.8347417840375587, 'f1': 0.7980251346499103, 'number': 1065} | 0.7118 | 0.7908 | 0.7492 | 0.8004 |
| 0.3126 | 12.0 | 120 | 0.6896 | {'precision': 0.697198275862069, 'recall': 0.799752781211372, 'f1': 0.7449625791594704, 'number': 809} | {'precision': 0.36283185840707965, 'recall': 0.3445378151260504, 'f1': 0.35344827586206895, 'number': 119} | {'precision': 0.7788632326820604, 'recall': 0.8234741784037559, 'f1': 0.8005476951163851, 'number': 1065} | 0.7222 | 0.7852 | 0.7524 | 0.8012 |
| 0.2979 | 13.0 | 130 | 0.6997 | {'precision': 0.6992399565689468, 'recall': 0.796044499381953, 'f1': 0.7445086705202313, 'number': 809} | {'precision': 0.3416666666666667, 'recall': 0.3445378151260504, 'f1': 0.34309623430962344, 'number': 119} | {'precision': 0.7763157894736842, 'recall': 0.8309859154929577, 'f1': 0.802721088435374, 'number': 1065} | 0.7199 | 0.7878 | 0.7523 | 0.8007 |
| 0.2712 | 14.0 | 140 | 0.7039 | {'precision': 0.7083333333333334, 'recall': 0.7985166872682324, 'f1': 0.7507263219058687, 'number': 809} | {'precision': 0.336, 'recall': 0.35294117647058826, 'f1': 0.3442622950819672, 'number': 119} | {'precision': 0.7771929824561403, 'recall': 0.831924882629108, 'f1': 0.8036281179138323, 'number': 1065} | 0.7230 | 0.7898 | 0.7549 | 0.8028 |
| 0.2738 | 15.0 | 150 | 0.7055 | {'precision': 0.7035830618892508, 'recall': 0.8009888751545118, 'f1': 0.7491329479768787, 'number': 809} | {'precision': 0.34146341463414637, 'recall': 0.35294117647058826, 'f1': 0.34710743801652894, 'number': 119} | {'precision': 0.7775816416593115, 'recall': 0.8272300469483568, 'f1': 0.8016378525932666, 'number': 1065} | 0.7216 | 0.7883 | 0.7535 | 0.8028 |
Framework versions
- Transformers 4.41.1
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1