lilt-en-funsd-layoutlmv3

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.6323
• Answer: {'precision': 0.8683901292596945, 'recall': 0.9045287637698899, 'f1': 0.8860911270983215, 'number': 817}
• Header: {'precision': 0.6095238095238096, 'recall': 0.5378151260504201, 'f1': 0.5714285714285715, 'number': 119}
• Question: {'precision': 0.90063233965673, 'recall': 0.9257195914577531, 'f1': 0.913003663003663, 'number': 1077}
• Overall Precision: 0.8725
• Overall Recall: 0.8942
• Overall F1: 0.8832
• Overall Accuracy: 0.8071

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.4052	10.5263	200	1.0646	{'precision': 0.8046709129511678, 'recall': 0.9277845777233782, 'f1': 0.861853325753269, 'number': 817}	{'precision': 0.5803571428571429, 'recall': 0.5462184873949579, 'f1': 0.5627705627705628, 'number': 119}	{'precision': 0.8797061524334252, 'recall': 0.8895078922934077, 'f1': 0.8845798707294553, 'number': 1077}	0.8311	0.8847	0.8571	0.7850
0.0474	21.0526	400	1.2300	{'precision': 0.8500590318772137, 'recall': 0.8812729498164015, 'f1': 0.8653846153846153, 'number': 817}	{'precision': 0.5454545454545454, 'recall': 0.6554621848739496, 'f1': 0.5954198473282444, 'number': 119}	{'precision': 0.8798908098271155, 'recall': 0.8978644382544104, 'f1': 0.8887867647058824, 'number': 1077}	0.8449	0.8768	0.8606	0.8026
0.0127	31.5789	600	1.5767	{'precision': 0.8359728506787331, 'recall': 0.9045287637698899, 'f1': 0.8689006466784246, 'number': 817}	{'precision': 0.5583333333333333, 'recall': 0.5630252100840336, 'f1': 0.5606694560669456, 'number': 119}	{'precision': 0.8940397350993378, 'recall': 0.8774373259052924, 'f1': 0.8856607310215557, 'number': 1077}	0.8496	0.8698	0.8596	0.7835
0.0085	42.1053	800	1.3875	{'precision': 0.833710407239819, 'recall': 0.9020807833537332, 'f1': 0.8665490887713109, 'number': 817}	{'precision': 0.6363636363636364, 'recall': 0.5294117647058824, 'f1': 0.5779816513761468, 'number': 119}	{'precision': 0.8825654923215899, 'recall': 0.9071494893221913, 'f1': 0.8946886446886446, 'number': 1077}	0.8502	0.8828	0.8662	0.8072
0.0058	52.6316	1000	1.4794	{'precision': 0.8272017837235228, 'recall': 0.9082007343941249, 'f1': 0.8658109684947491, 'number': 817}	{'precision': 0.5203252032520326, 'recall': 0.5378151260504201, 'f1': 0.5289256198347108, 'number': 119}	{'precision': 0.8829981718464351, 'recall': 0.8969359331476323, 'f1': 0.889912482726854, 'number': 1077}	0.8382	0.8803	0.8587	0.7964
0.0038	63.1579	1200	1.5286	{'precision': 0.8443935926773455, 'recall': 0.9033047735618115, 'f1': 0.872856298048492, 'number': 817}	{'precision': 0.625, 'recall': 0.5042016806722689, 'f1': 0.5581395348837209, 'number': 119}	{'precision': 0.8991674375578168, 'recall': 0.9025069637883009, 'f1': 0.9008341056533827, 'number': 1077}	0.8630	0.8793	0.8711	0.8084
0.0023	73.6842	1400	1.6443	{'precision': 0.8725146198830409, 'recall': 0.9130966952264382, 'f1': 0.8923444976076554, 'number': 817}	{'precision': 0.5, 'recall': 0.5210084033613446, 'f1': 0.5102880658436215, 'number': 119}	{'precision': 0.8906955736224029, 'recall': 0.9155060352831941, 'f1': 0.9029304029304029, 'number': 1077}	0.8600	0.8912	0.8753	0.8054
0.0012	84.2105	1600	1.6379	{'precision': 0.8404977375565611, 'recall': 0.9094247246022031, 'f1': 0.8736037624926513, 'number': 817}	{'precision': 0.6224489795918368, 'recall': 0.5126050420168067, 'f1': 0.5622119815668203, 'number': 119}	{'precision': 0.8944444444444445, 'recall': 0.8969359331476323, 'f1': 0.8956884561891516, 'number': 1077}	0.8584	0.8793	0.8687	0.8008
0.0005	94.7368	1800	1.6798	{'precision': 0.8450057405281286, 'recall': 0.9008567931456548, 'f1': 0.8720379146919431, 'number': 817}	{'precision': 0.6534653465346535, 'recall': 0.5546218487394958, 'f1': 0.6000000000000001, 'number': 119}	{'precision': 0.8886861313868614, 'recall': 0.904363974001857, 'f1': 0.8964565117349288, 'number': 1077}	0.8588	0.8823	0.8704	0.7988
0.0004	105.2632	2000	1.6804	{'precision': 0.8596491228070176, 'recall': 0.8996328029375765, 'f1': 0.8791866028708135, 'number': 817}	{'precision': 0.5203252032520326, 'recall': 0.5378151260504201, 'f1': 0.5289256198347108, 'number': 119}	{'precision': 0.8869801084990958, 'recall': 0.9108635097493036, 'f1': 0.8987631699496106, 'number': 1077}	0.8541	0.8843	0.8689	0.8032
0.0003	115.7895	2200	1.6352	{'precision': 0.8713105076741441, 'recall': 0.9033047735618115, 'f1': 0.8870192307692307, 'number': 817}	{'precision': 0.6153846153846154, 'recall': 0.5378151260504201, 'f1': 0.5739910313901345, 'number': 119}	{'precision': 0.8938848920863309, 'recall': 0.9229340761374187, 'f1': 0.9081772498857925, 'number': 1077}	0.8706	0.8922	0.8813	0.8066
0.0002	126.3158	2400	1.6323	{'precision': 0.8683901292596945, 'recall': 0.9045287637698899, 'f1': 0.8860911270983215, 'number': 817}	{'precision': 0.6095238095238096, 'recall': 0.5378151260504201, 'f1': 0.5714285714285715, 'number': 119}	{'precision': 0.90063233965673, 'recall': 0.9257195914577531, 'f1': 0.913003663003663, 'number': 1077}	0.8725	0.8942	0.8832	0.8071

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1