layoutlm-funsd3

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6649
• Answer: {'precision': 0.6862955032119914, 'recall': 0.792336217552534, 'f1': 0.7355134825014343, 'number': 809}
• Header: {'precision': 0.2782608695652174, 'recall': 0.2689075630252101, 'f1': 0.2735042735042735, 'number': 119}
• Question: {'precision': 0.731418918918919, 'recall': 0.8131455399061033, 'f1': 0.7701200533570476, 'number': 1065}
• Overall Precision: 0.6892
• Overall Recall: 0.7722
• Overall F1: 0.7283
• Overall Accuracy: 0.8077

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 25
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.9006	1.0	10	1.9148	{'precision': 0.034013605442176874, 'recall': 0.11742892459826947, 'f1': 0.05274847307051638, 'number': 809}	{'precision': 0.007547169811320755, 'recall': 0.01680672268907563, 'f1': 0.010416666666666666, 'number': 119}	{'precision': 0.035897435897435895, 'recall': 0.06572769953051644, 'f1': 0.04643449419568822, 'number': 1065}	0.0333	0.0838	0.0477	0.1975
1.8905	2.0	20	1.9045	{'precision': 0.03486238532110092, 'recall': 0.11742892459826947, 'f1': 0.05376344086021505, 'number': 809}	{'precision': 0.004761904761904762, 'recall': 0.008403361344537815, 'f1': 0.006079027355623101, 'number': 119}	{'precision': 0.03635432667690732, 'recall': 0.06666666666666667, 'f1': 0.04705102717031146, 'number': 1065}	0.0342	0.0838	0.0485	0.2074
1.8811	3.0	30	1.8873	{'precision': 0.032742681047765794, 'recall': 0.10506798516687268, 'f1': 0.0499265785609398, 'number': 809}	{'precision': 0.00684931506849315, 'recall': 0.008403361344537815, 'f1': 0.007547169811320755, 'number': 119}	{'precision': 0.03967027305512622, 'recall': 0.07230046948356808, 'f1': 0.051230871590153035, 'number': 1065}	0.0348	0.0818	0.0488	0.2231
1.8598	4.0	40	1.8641	{'precision': 0.029242174629324547, 'recall': 0.08776266996291718, 'f1': 0.043867778807537845, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.045021186440677964, 'recall': 0.07981220657276995, 'f1': 0.057568574331188616, 'number': 1065}	0.0355	0.0783	0.0489	0.2434
1.8352	5.0	50	1.8359	{'precision': 0.027777777777777776, 'recall': 0.07416563658838071, 'f1': 0.040417649040080834, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.05329512893982808, 'recall': 0.08732394366197183, 'f1': 0.06619217081850534, 'number': 1065}	0.0389	0.0768	0.0516	0.2684
1.805	6.0	60	1.8038	{'precision': 0.021965952773201538, 'recall': 0.049443757725587144, 'f1': 0.030418250950570342, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.056172436316133244, 'recall': 0.08075117370892018, 'f1': 0.06625577812018489, 'number': 1065}	0.0374	0.0632	0.0470	0.2894
1.7726	7.0	70	1.7692	{'precision': 0.02216252518468771, 'recall': 0.0407911001236094, 'f1': 0.028720626631853784, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.06599378881987578, 'recall': 0.07981220657276995, 'f1': 0.07224819379515512, 'number': 1065}	0.0424	0.0592	0.0494	0.3088
1.7332	8.0	80	1.7277	{'precision': 0.018210609659540775, 'recall': 0.02843016069221261, 'f1': 0.022200772200772198, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.08497316636851521, 'recall': 0.0892018779342723, 'f1': 0.08703618873110398, 'number': 1065}	0.0496	0.0592	0.0540	0.3301
1.6941	9.0	90	1.6821	{'precision': 0.024411508282476024, 'recall': 0.034610630407911, 'f1': 0.028629856850715743, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.11352885525070956, 'recall': 0.11267605633802817, 'f1': 0.11310084825636192, 'number': 1065}	0.0672	0.0743	0.0705	0.3529
1.6579	10.0	100	1.6290	{'precision': 0.03211805555555555, 'recall': 0.04573547589616811, 'f1': 0.03773584905660377, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.1650902837489252, 'recall': 0.18028169014084508, 'f1': 0.17235188509874327, 'number': 1065}	0.0989	0.1149	0.1063	0.3897
1.5882	11.0	110	1.5600	{'precision': 0.06073943661971831, 'recall': 0.08529048207663782, 'f1': 0.07095115681233934, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.22230538922155688, 'recall': 0.27887323943661974, 'f1': 0.2473969179508538, 'number': 1065}	0.1481	0.1836	0.1639	0.4485
1.5164	12.0	120	1.4778	{'precision': 0.111, 'recall': 0.13720642768850433, 'f1': 0.12271973466003316, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.28520877565463554, 'recall': 0.3784037558685446, 'f1': 0.3252623083131558, 'number': 1065}	0.2130	0.2579	0.2333	0.5018
1.4203	13.0	130	1.3796	{'precision': 0.1891304347826087, 'recall': 0.21508034610630408, 'f1': 0.20127241179872757, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.37259100642398285, 'recall': 0.49014084507042255, 'f1': 0.4233576642335766, 'number': 1065}	0.2999	0.3492	0.3227	0.5474
1.2916	14.0	140	1.2617	{'precision': 0.27813852813852813, 'recall': 0.3176761433868974, 'f1': 0.29659549913444894, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4349112426035503, 'recall': 0.5521126760563381, 'f1': 0.48655357881671496, 'number': 1065}	0.3713	0.4240	0.3959	0.5943
1.1747	15.0	150	1.1279	{'precision': 0.3726775956284153, 'recall': 0.4215080346106304, 'f1': 0.39559164733178653, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5030816640986132, 'recall': 0.6131455399061033, 'f1': 0.5526872619551417, 'number': 1065}	0.4482	0.4987	0.4721	0.6467
1.0441	16.0	160	0.9940	{'precision': 0.46846846846846846, 'recall': 0.5784919653893696, 'f1': 0.5176991150442478, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5588709677419355, 'recall': 0.6507042253521127, 'f1': 0.6013015184381778, 'number': 1065}	0.5126	0.5825	0.5453	0.7049
0.9042	17.0	170	0.8718	{'precision': 0.567703109327984, 'recall': 0.6996291718170581, 'f1': 0.6267995570321152, 'number': 809}	{'precision': 0.020833333333333332, 'recall': 0.008403361344537815, 'f1': 0.011976047904191616, 'number': 119}	{'precision': 0.6192468619246861, 'recall': 0.6948356807511737, 'f1': 0.6548672566371683, 'number': 1065}	0.5835	0.6558	0.6175	0.7473
0.7845	18.0	180	0.7760	{'precision': 0.597478176527643, 'recall': 0.761433868974042, 'f1': 0.6695652173913043, 'number': 809}	{'precision': 0.16981132075471697, 'recall': 0.07563025210084033, 'f1': 0.10465116279069768, 'number': 119}	{'precision': 0.6678082191780822, 'recall': 0.7323943661971831, 'f1': 0.6986117330944918, 'number': 1065}	0.6239	0.7050	0.6620	0.7693
0.7023	19.0	190	0.7265	{'precision': 0.619188921859545, 'recall': 0.7737948084054388, 'f1': 0.6879120879120879, 'number': 809}	{'precision': 0.22580645161290322, 'recall': 0.11764705882352941, 'f1': 0.15469613259668508, 'number': 119}	{'precision': 0.6943722943722944, 'recall': 0.7530516431924883, 'f1': 0.7225225225225225, 'number': 1065}	0.6472	0.7235	0.6833	0.7783
0.6331	20.0	200	0.7139	{'precision': 0.6457446808510638, 'recall': 0.7503090234857849, 'f1': 0.6941109205260149, 'number': 809}	{'precision': 0.25609756097560976, 'recall': 0.17647058823529413, 'f1': 0.208955223880597, 'number': 119}	{'precision': 0.6934548467274234, 'recall': 0.7859154929577464, 'f1': 0.7367957746478873, 'number': 1065}	0.6572	0.7351	0.6940	0.7900
0.5789	21.0	210	0.6960	{'precision': 0.6496815286624203, 'recall': 0.7564894932014833, 'f1': 0.6990291262135921, 'number': 809}	{'precision': 0.25274725274725274, 'recall': 0.19327731092436976, 'f1': 0.21904761904761905, 'number': 119}	{'precision': 0.706081081081081, 'recall': 0.7849765258215963, 'f1': 0.7434415295686971, 'number': 1065}	0.6635	0.7381	0.6988	0.7929
0.5417	22.0	220	0.6774	{'precision': 0.6699346405228758, 'recall': 0.7601977750309024, 'f1': 0.7122177185871453, 'number': 809}	{'precision': 0.211864406779661, 'recall': 0.21008403361344538, 'f1': 0.2109704641350211, 'number': 119}	{'precision': 0.6981907894736842, 'recall': 0.7971830985915493, 'f1': 0.7444103463393249, 'number': 1065}	0.6612	0.7471	0.7015	0.7959
0.481	23.0	230	0.6671	{'precision': 0.6748400852878464, 'recall': 0.7824474660074165, 'f1': 0.7246708643388666, 'number': 809}	{'precision': 0.2540983606557377, 'recall': 0.2605042016806723, 'f1': 0.2572614107883818, 'number': 119}	{'precision': 0.718013468013468, 'recall': 0.8009389671361502, 'f1': 0.7572126054150022, 'number': 1065}	0.6748	0.7612	0.7154	0.8022
0.4419	24.0	240	0.6534	{'precision': 0.6799140708915145, 'recall': 0.7824474660074165, 'f1': 0.7275862068965516, 'number': 809}	{'precision': 0.2818181818181818, 'recall': 0.2605042016806723, 'f1': 0.27074235807860264, 'number': 119}	{'precision': 0.7332185886402753, 'recall': 0.8, 'f1': 0.7651549169286035, 'number': 1065}	0.6882	0.7607	0.7226	0.8054
0.406	25.0	250	0.6649	{'precision': 0.6862955032119914, 'recall': 0.792336217552534, 'f1': 0.7355134825014343, 'number': 809}	{'precision': 0.2782608695652174, 'recall': 0.2689075630252101, 'f1': 0.2735042735042735, 'number': 119}	{'precision': 0.731418918918919, 'recall': 0.8131455399061033, 'f1': 0.7701200533570476, 'number': 1065}	0.6892	0.7722	0.7283	0.8077

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1