layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 1.1516
• Answer: {'precision': 0.38400702987697716, 'recall': 0.5401730531520396, 'f1': 0.44889573703133023, 'number': 809}
• Header: {'precision': 0.3218390804597701, 'recall': 0.23529411764705882, 'f1': 0.27184466019417475, 'number': 119}
• Question: {'precision': 0.5132192846034215, 'recall': 0.6197183098591549, 'f1': 0.5614632071458954, 'number': 1065}
• Overall Precision: 0.4480
• Overall Recall: 0.5645
• Overall F1: 0.4996
• Overall Accuracy: 0.6209

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7219	1.0	10	1.5555	{'precision': 0.04431137724550898, 'recall': 0.04573547589616811, 'f1': 0.04501216545012165, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.26320501342882724, 'recall': 0.27605633802816903, 'f1': 0.2694775435380385, 'number': 1065}	0.1696	0.1661	0.1678	0.3589
1.4917	2.0	20	1.3323	{'precision': 0.17622377622377622, 'recall': 0.311495673671199, 'f1': 0.2251004912907548, 'number': 809}	{'precision': 0.1, 'recall': 0.008403361344537815, 'f1': 0.015503875968992248, 'number': 119}	{'precision': 0.29382407985028075, 'recall': 0.4422535211267606, 'f1': 0.3530734632683658, 'number': 1065}	0.2379	0.3633	0.2875	0.4413
1.2799	3.0	30	1.2482	{'precision': 0.24236517218973358, 'recall': 0.4610630407911001, 'f1': 0.317717206132879, 'number': 809}	{'precision': 0.273972602739726, 'recall': 0.16806722689075632, 'f1': 0.20833333333333331, 'number': 119}	{'precision': 0.35655737704918034, 'recall': 0.4084507042253521, 'f1': 0.38074398249452956, 'number': 1065}	0.2924	0.4155	0.3432	0.4580
1.1477	4.0	40	1.1758	{'precision': 0.2900516795865633, 'recall': 0.5550061804697157, 'f1': 0.38099278744166315, 'number': 809}	{'precision': 0.3559322033898305, 'recall': 0.17647058823529413, 'f1': 0.2359550561797753, 'number': 119}	{'precision': 0.4393939393939394, 'recall': 0.49014084507042255, 'f1': 0.46338215712383485, 'number': 1065}	0.3549	0.4977	0.4144	0.5219
1.0484	5.0	50	1.0885	{'precision': 0.3271441202475685, 'recall': 0.4573547589616811, 'f1': 0.3814432989690722, 'number': 809}	{'precision': 0.2826086956521739, 'recall': 0.2184873949579832, 'f1': 0.24644549763033172, 'number': 119}	{'precision': 0.4808, 'recall': 0.564319248826291, 'f1': 0.5192224622030237, 'number': 1065}	0.4032	0.5003	0.4465	0.5827
0.9672	6.0	60	1.0745	{'precision': 0.30431309904153353, 'recall': 0.47095179233621753, 'f1': 0.36972343522561857, 'number': 809}	{'precision': 0.34782608695652173, 'recall': 0.20168067226890757, 'f1': 0.25531914893617025, 'number': 119}	{'precision': 0.43936243936243935, 'recall': 0.5953051643192488, 'f1': 0.5055821371610846, 'number': 1065}	0.3759	0.5213	0.4368	0.5916
0.8787	7.0	70	1.1863	{'precision': 0.3697033898305085, 'recall': 0.43139678615574784, 'f1': 0.3981745579007416, 'number': 809}	{'precision': 0.25, 'recall': 0.2184873949579832, 'f1': 0.23318385650224216, 'number': 119}	{'precision': 0.4801556420233463, 'recall': 0.5793427230046948, 'f1': 0.5251063829787234, 'number': 1065}	0.4252	0.4977	0.4586	0.5870
0.8501	8.0	80	1.1043	{'precision': 0.31553860819828405, 'recall': 0.40914709517923364, 'f1': 0.3562970936490851, 'number': 809}	{'precision': 0.3484848484848485, 'recall': 0.19327731092436976, 'f1': 0.24864864864864866, 'number': 119}	{'precision': 0.41997593261131166, 'recall': 0.6553990610328638, 'f1': 0.5119178584525119, 'number': 1065}	0.3788	0.5278	0.4411	0.5878
0.805	9.0	90	1.0872	{'precision': 0.3356828193832599, 'recall': 0.47095179233621753, 'f1': 0.39197530864197533, 'number': 809}	{'precision': 0.32894736842105265, 'recall': 0.21008403361344538, 'f1': 0.25641025641025644, 'number': 119}	{'precision': 0.45454545454545453, 'recall': 0.6197183098591549, 'f1': 0.5244338498212157, 'number': 1065}	0.4003	0.5349	0.4579	0.6053
0.7686	10.0	100	1.1006	{'precision': 0.35418427726120033, 'recall': 0.5179233621755254, 'f1': 0.42068273092369474, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.2184873949579832, 'f1': 0.2639593908629441, 'number': 119}	{'precision': 0.49634443541835904, 'recall': 0.5737089201877934, 'f1': 0.5322299651567944, 'number': 1065}	0.4238	0.5299	0.4709	0.6028
0.7078	11.0	110	1.1631	{'precision': 0.38475665748393023, 'recall': 0.5179233621755254, 'f1': 0.4415173867228662, 'number': 809}	{'precision': 0.28846153846153844, 'recall': 0.25210084033613445, 'f1': 0.26905829596412556, 'number': 119}	{'precision': 0.520764119601329, 'recall': 0.5887323943661972, 'f1': 0.5526663728514765, 'number': 1065}	0.4489	0.5399	0.4902	0.6064
0.7162	12.0	120	1.1517	{'precision': 0.36400817995910023, 'recall': 0.4400494437577256, 'f1': 0.3984331281477337, 'number': 809}	{'precision': 0.28421052631578947, 'recall': 0.226890756302521, 'f1': 0.25233644859813087, 'number': 119}	{'precision': 0.4661458333333333, 'recall': 0.672300469483568, 'f1': 0.550557477893118, 'number': 1065}	0.4212	0.5514	0.4776	0.6014
0.6912	13.0	130	1.2013	{'precision': 0.3880718954248366, 'recall': 0.5871446229913473, 'f1': 0.4672897196261682, 'number': 809}	{'precision': 0.3888888888888889, 'recall': 0.23529411764705882, 'f1': 0.2931937172774869, 'number': 119}	{'precision': 0.5526552655265526, 'recall': 0.5765258215962441, 'f1': 0.5643382352941176, 'number': 1065}	0.4641	0.5605	0.5077	0.6082
0.664	14.0	140	1.1337	{'precision': 0.37344028520499106, 'recall': 0.5179233621755254, 'f1': 0.4339720352149145, 'number': 809}	{'precision': 0.3218390804597701, 'recall': 0.23529411764705882, 'f1': 0.27184466019417475, 'number': 119}	{'precision': 0.5037650602409639, 'recall': 0.6281690140845071, 'f1': 0.5591307981613038, 'number': 1065}	0.4399	0.5600	0.4927	0.6142
0.6496	15.0	150	1.1516	{'precision': 0.38400702987697716, 'recall': 0.5401730531520396, 'f1': 0.44889573703133023, 'number': 809}	{'precision': 0.3218390804597701, 'recall': 0.23529411764705882, 'f1': 0.27184466019417475, 'number': 119}	{'precision': 0.5132192846034215, 'recall': 0.6197183098591549, 'f1': 0.5614632071458954, 'number': 1065}	0.4480	0.5645	0.4996	0.6209

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2