layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 1.1050
• Answer: {'precision': 0.37133808392715756, 'recall': 0.5797280593325093, 'f1': 0.45270270270270274, 'number': 809}
• Header: {'precision': 0.32926829268292684, 'recall': 0.226890756302521, 'f1': 0.26865671641791045, 'number': 119}
• Question: {'precision': 0.49682539682539684, 'recall': 0.5877934272300469, 'f1': 0.538494623655914, 'number': 1065}
• Overall Precision: 0.4307
• Overall Recall: 0.5630
• Overall F1: 0.4880
• Overall Accuracy: 0.6093

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8038	1.0	10	1.5073	{'precision': 0.06441476826394343, 'recall': 0.10135970333745364, 'f1': 0.07877041306436118, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.24326241134751772, 'recall': 0.3220657276995305, 'f1': 0.2771717171717171, 'number': 1065}	0.1584	0.2132	0.1818	0.3843
1.4521	2.0	20	1.3396	{'precision': 0.20421753607103219, 'recall': 0.45488257107540175, 'f1': 0.28188433550363845, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.2649350649350649, 'recall': 0.38309859154929576, 'f1': 0.31324376199616116, 'number': 1065}	0.2321	0.3894	0.2909	0.4184
1.278	3.0	30	1.2050	{'precision': 0.2645794966236955, 'recall': 0.5327564894932015, 'f1': 0.3535684987694832, 'number': 809}	{'precision': 0.12903225806451613, 'recall': 0.06722689075630252, 'f1': 0.08839779005524862, 'number': 119}	{'precision': 0.34989503149055284, 'recall': 0.4694835680751174, 'f1': 0.400962309542903, 'number': 1065}	0.3010	0.4711	0.3673	0.4760
1.1503	4.0	40	1.1044	{'precision': 0.28089080459770116, 'recall': 0.48331273176761436, 'f1': 0.3552930486142663, 'number': 809}	{'precision': 0.2391304347826087, 'recall': 0.18487394957983194, 'f1': 0.2085308056872038, 'number': 119}	{'precision': 0.4, 'recall': 0.5295774647887324, 'f1': 0.45575757575757575, 'number': 1065}	0.3376	0.4902	0.3998	0.5630
1.07	5.0	50	1.1546	{'precision': 0.30014025245441794, 'recall': 0.5290482076637825, 'f1': 0.38299776286353465, 'number': 809}	{'precision': 0.3188405797101449, 'recall': 0.18487394957983194, 'f1': 0.23404255319148937, 'number': 119}	{'precision': 0.4058373870743572, 'recall': 0.5483568075117371, 'f1': 0.4664536741214057, 'number': 1065}	0.3524	0.5188	0.4197	0.5383
0.9914	6.0	60	1.0507	{'precision': 0.3119065010956903, 'recall': 0.5278121137206427, 'f1': 0.3921028466483012, 'number': 809}	{'precision': 0.2345679012345679, 'recall': 0.15966386554621848, 'f1': 0.18999999999999997, 'number': 119}	{'precision': 0.4122938530734633, 'recall': 0.5164319248826291, 'f1': 0.45852438516048355, 'number': 1065}	0.3578	0.4997	0.4170	0.6002
0.9373	7.0	70	1.0652	{'precision': 0.3710691823899371, 'recall': 0.43757725587144625, 'f1': 0.4015882019285309, 'number': 809}	{'precision': 0.25510204081632654, 'recall': 0.21008403361344538, 'f1': 0.23041474654377883, 'number': 119}	{'precision': 0.4739583333333333, 'recall': 0.5981220657276995, 'f1': 0.5288501452885015, 'number': 1065}	0.4240	0.5098	0.4630	0.6006
0.8833	8.0	80	1.0389	{'precision': 0.3351605324980423, 'recall': 0.5290482076637825, 'f1': 0.4103547459252157, 'number': 809}	{'precision': 0.375, 'recall': 0.20168067226890757, 'f1': 0.2622950819672132, 'number': 119}	{'precision': 0.44528301886792454, 'recall': 0.5539906103286385, 'f1': 0.49372384937238495, 'number': 1065}	0.3908	0.5228	0.4473	0.6143
0.8029	9.0	90	1.0520	{'precision': 0.3685612788632327, 'recall': 0.5129789864029666, 'f1': 0.4289405684754522, 'number': 809}	{'precision': 0.28695652173913044, 'recall': 0.2773109243697479, 'f1': 0.2820512820512821, 'number': 119}	{'precision': 0.4902874902874903, 'recall': 0.5924882629107981, 'f1': 0.5365646258503401, 'number': 1065}	0.4268	0.5414	0.4773	0.6023
0.7658	10.0	100	1.0764	{'precision': 0.3386511965192168, 'recall': 0.5772558714462299, 'f1': 0.42687385740402195, 'number': 809}	{'precision': 0.3709677419354839, 'recall': 0.19327731092436976, 'f1': 0.2541436464088398, 'number': 119}	{'precision': 0.4847986852917009, 'recall': 0.5539906103286385, 'f1': 0.5170902716914987, 'number': 1065}	0.4063	0.5419	0.4644	0.6066
0.7112	11.0	110	1.0675	{'precision': 0.3728963684676705, 'recall': 0.5203955500618047, 'f1': 0.43446852425180593, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.21008403361344538, 'f1': 0.2577319587628866, 'number': 119}	{'precision': 0.4918032786885246, 'recall': 0.5915492957746479, 'f1': 0.5370843989769821, 'number': 1065}	0.4330	0.5399	0.4806	0.6124
0.6875	12.0	120	1.1100	{'precision': 0.37746256895193064, 'recall': 0.5920889987639061, 'f1': 0.46102021174205965, 'number': 809}	{'precision': 0.33783783783783783, 'recall': 0.21008403361344538, 'f1': 0.25906735751295334, 'number': 119}	{'precision': 0.514554794520548, 'recall': 0.564319248826291, 'f1': 0.5382892969099866, 'number': 1065}	0.4401	0.5544	0.4907	0.6102
0.6571	13.0	130	1.0804	{'precision': 0.36231884057971014, 'recall': 0.5253399258343634, 'f1': 0.4288597376387487, 'number': 809}	{'precision': 0.313953488372093, 'recall': 0.226890756302521, 'f1': 0.2634146341463415, 'number': 119}	{'precision': 0.46940244780417567, 'recall': 0.612206572769953, 'f1': 0.5313773431132844, 'number': 1065}	0.4169	0.5539	0.4758	0.6141
0.6564	14.0	140	1.0934	{'precision': 0.37943262411347517, 'recall': 0.5290482076637825, 'f1': 0.44192049561177077, 'number': 809}	{'precision': 0.37662337662337664, 'recall': 0.24369747899159663, 'f1': 0.29591836734693877, 'number': 119}	{'precision': 0.49803613511390415, 'recall': 0.5953051643192488, 'f1': 0.542343883661249, 'number': 1065}	0.4403	0.5474	0.4880	0.6215
0.6558	15.0	150	1.1050	{'precision': 0.37133808392715756, 'recall': 0.5797280593325093, 'f1': 0.45270270270270274, 'number': 809}	{'precision': 0.32926829268292684, 'recall': 0.226890756302521, 'f1': 0.26865671641791045, 'number': 119}	{'precision': 0.49682539682539684, 'recall': 0.5877934272300469, 'f1': 0.538494623655914, 'number': 1065}	0.4307	0.5630	0.4880	0.6093

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.18.0
• Tokenizers 0.15.0