layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6843
• Answer: {'precision': 0.6938997821350763, 'recall': 0.7873918417799752, 'f1': 0.7376954255935148, 'number': 809}
• Header: {'precision': 0.27941176470588236, 'recall': 0.31932773109243695, 'f1': 0.2980392156862745, 'number': 119}
• Question: {'precision': 0.7749562171628721, 'recall': 0.8309859154929577, 'f1': 0.8019936565473492, 'number': 1065}
• Overall Precision: 0.7104
• Overall Recall: 0.7827
• Overall F1: 0.7448
• Overall Accuracy: 0.8076

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8035	1.0	10	1.6086	{'precision': 0.007142857142857143, 'recall': 0.003708281829419036, 'f1': 0.004882017900732303, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.07957559681697612, 'recall': 0.028169014084507043, 'f1': 0.04160887656033287, 'number': 1065}	0.0414	0.0166	0.0237	0.3175
1.4936	2.0	20	1.2735	{'precision': 0.279126213592233, 'recall': 0.4264524103831891, 'f1': 0.3374083129584352, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4406651549508692, 'recall': 0.5474178403755868, 'f1': 0.48827470686767166, 'number': 1065}	0.3626	0.4656	0.4077	0.6074
1.1259	3.0	30	0.9718	{'precision': 0.47892074198988194, 'recall': 0.7021013597033374, 'f1': 0.569423558897243, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5904121110176619, 'recall': 0.6591549295774648, 'f1': 0.6228926353149955, 'number': 1065}	0.5336	0.6372	0.5808	0.6760
0.8568	4.0	40	0.8421	{'precision': 0.5595126522961574, 'recall': 0.7379480840543882, 'f1': 0.6364605543710021, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.6669595782073814, 'recall': 0.7126760563380282, 'f1': 0.6890603722197005, 'number': 1065}	0.6059	0.6804	0.6410	0.7310
0.7275	5.0	50	0.7430	{'precision': 0.6401673640167364, 'recall': 0.7564894932014833, 'f1': 0.693484419263456, 'number': 809}	{'precision': 0.12643678160919541, 'recall': 0.09243697478991597, 'f1': 0.10679611650485439, 'number': 119}	{'precision': 0.6825, 'recall': 0.7690140845070422, 'f1': 0.7231788079470199, 'number': 1065}	0.6429	0.7235	0.6808	0.7727
0.6109	6.0	60	0.6965	{'precision': 0.6494736842105263, 'recall': 0.7626699629171817, 'f1': 0.7015349630471859, 'number': 809}	{'precision': 0.125, 'recall': 0.09243697478991597, 'f1': 0.10628019323671498, 'number': 119}	{'precision': 0.6780415430267063, 'recall': 0.8582159624413146, 'f1': 0.7575631993369251, 'number': 1065}	0.6463	0.7737	0.7043	0.7862
0.5341	7.0	70	0.6816	{'precision': 0.6745945945945946, 'recall': 0.7713226205191595, 'f1': 0.7197231833910035, 'number': 809}	{'precision': 0.22727272727272727, 'recall': 0.21008403361344538, 'f1': 0.21834061135371177, 'number': 119}	{'precision': 0.7435037720033529, 'recall': 0.8328638497652582, 'f1': 0.7856510186005314, 'number': 1065}	0.6894	0.7707	0.7278	0.7920
0.4811	8.0	80	0.6577	{'precision': 0.6800870511425462, 'recall': 0.7725587144622992, 'f1': 0.7233796296296297, 'number': 809}	{'precision': 0.2072072072072072, 'recall': 0.19327731092436976, 'f1': 0.2, 'number': 119}	{'precision': 0.7440132122213047, 'recall': 0.8460093896713615, 'f1': 0.7917398945518453, 'number': 1065}	0.6912	0.7772	0.7317	0.7986
0.4241	9.0	90	0.6586	{'precision': 0.6898454746136865, 'recall': 0.7725587144622992, 'f1': 0.7288629737609328, 'number': 809}	{'precision': 0.2535211267605634, 'recall': 0.3025210084033613, 'f1': 0.2758620689655173, 'number': 119}	{'precision': 0.751269035532995, 'recall': 0.8338028169014085, 'f1': 0.7903871829105474, 'number': 1065}	0.6946	0.7772	0.7336	0.7991
0.3784	10.0	100	0.6511	{'precision': 0.6879739978331527, 'recall': 0.7849196538936959, 'f1': 0.7332563510392609, 'number': 809}	{'precision': 0.2833333333333333, 'recall': 0.2857142857142857, 'f1': 0.2845188284518828, 'number': 119}	{'precision': 0.7590870667793744, 'recall': 0.8431924882629108, 'f1': 0.7989323843416369, 'number': 1065}	0.7040	0.7863	0.7428	0.8046
0.3425	11.0	110	0.6611	{'precision': 0.6975982532751092, 'recall': 0.7898640296662547, 'f1': 0.7408695652173912, 'number': 809}	{'precision': 0.26865671641791045, 'recall': 0.3025210084033613, 'f1': 0.2845849802371542, 'number': 119}	{'precision': 0.7737162750217581, 'recall': 0.8347417840375587, 'f1': 0.803071364046974, 'number': 1065}	0.7112	0.7847	0.7462	0.8116
0.3225	12.0	120	0.6676	{'precision': 0.6957470010905126, 'recall': 0.788627935723115, 'f1': 0.7392815758980301, 'number': 809}	{'precision': 0.27941176470588236, 'recall': 0.31932773109243695, 'f1': 0.2980392156862745, 'number': 119}	{'precision': 0.7806167400881058, 'recall': 0.831924882629108, 'f1': 0.8054545454545454, 'number': 1065}	0.7139	0.7837	0.7472	0.8081
0.302	13.0	130	0.6698	{'precision': 0.6956043956043956, 'recall': 0.7824474660074165, 'f1': 0.7364746945898778, 'number': 809}	{'precision': 0.2878787878787879, 'recall': 0.31932773109243695, 'f1': 0.302788844621514, 'number': 119}	{'precision': 0.7730434782608696, 'recall': 0.8347417840375587, 'f1': 0.8027088036117382, 'number': 1065}	0.7117	0.7827	0.7455	0.8133
0.2915	14.0	140	0.6845	{'precision': 0.6978260869565217, 'recall': 0.7935723114956736, 'f1': 0.742625795257374, 'number': 809}	{'precision': 0.2857142857142857, 'recall': 0.31932773109243695, 'f1': 0.30158730158730157, 'number': 119}	{'precision': 0.7771929824561403, 'recall': 0.831924882629108, 'f1': 0.8036281179138323, 'number': 1065}	0.7141	0.7858	0.7482	0.8052
0.2872	15.0	150	0.6843	{'precision': 0.6938997821350763, 'recall': 0.7873918417799752, 'f1': 0.7376954255935148, 'number': 809}	{'precision': 0.27941176470588236, 'recall': 0.31932773109243695, 'f1': 0.2980392156862745, 'number': 119}	{'precision': 0.7749562171628721, 'recall': 0.8309859154929577, 'f1': 0.8019936565473492, 'number': 1065}	0.7104	0.7827	0.7448	0.8076

Framework versions

• Transformers 4.22.2
• Pytorch 1.12.1+cu113
• Datasets 2.5.2
• Tokenizers 0.12.1