lilt-en-funsd

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5924
• Answer: {'precision': 0.8748538011695907, 'recall': 0.9155446756425949, 'f1': 0.8947368421052633, 'number': 817}
• Header: {'precision': 0.64, 'recall': 0.5378151260504201, 'f1': 0.5844748858447488, 'number': 119}
• Question: {'precision': 0.8945487042001787, 'recall': 0.9294336118848654, 'f1': 0.9116575591985429, 'number': 1077}
• Overall Precision: 0.8742
• Overall Recall: 0.9006
• Overall F1: 0.8872
• Overall Accuracy: 0.8193

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.3657	10.5263	200	0.9521	{'precision': 0.8126361655773421, 'recall': 0.9130966952264382, 'f1': 0.859942363112392, 'number': 817}	{'precision': 0.5252525252525253, 'recall': 0.4369747899159664, 'f1': 0.47706422018348627, 'number': 119}	{'precision': 0.8796046720575023, 'recall': 0.9090064995357474, 'f1': 0.8940639269406392, 'number': 1077}	0.8343	0.8828	0.8578	0.8059
0.0448	21.0526	400	1.2063	{'precision': 0.8845686512758202, 'recall': 0.8910648714810282, 'f1': 0.8878048780487805, 'number': 817}	{'precision': 0.5034965034965035, 'recall': 0.6050420168067226, 'f1': 0.549618320610687, 'number': 119}	{'precision': 0.8940092165898618, 'recall': 0.9006499535747446, 'f1': 0.8973172987974098, 'number': 1077}	0.8630	0.8793	0.8711	0.8133
0.0135	31.5789	600	1.3466	{'precision': 0.8726190476190476, 'recall': 0.8971848225214198, 'f1': 0.8847314423657212, 'number': 817}	{'precision': 0.4900662251655629, 'recall': 0.6218487394957983, 'f1': 0.5481481481481482, 'number': 119}	{'precision': 0.8789808917197452, 'recall': 0.8969359331476323, 'f1': 0.8878676470588236, 'number': 1077}	0.8483	0.8808	0.8642	0.8083
0.0069	42.1053	800	1.3562	{'precision': 0.8235294117647058, 'recall': 0.9082007343941249, 'f1': 0.8637951105937136, 'number': 817}	{'precision': 0.6413043478260869, 'recall': 0.4957983193277311, 'f1': 0.5592417061611374, 'number': 119}	{'precision': 0.8723981900452489, 'recall': 0.8950789229340761, 'f1': 0.8835930339138405, 'number': 1077}	0.8413	0.8768	0.8587	0.8063
0.0058	52.6316	1000	1.4131	{'precision': 0.8688524590163934, 'recall': 0.9082007343941249, 'f1': 0.8880909634949131, 'number': 817}	{'precision': 0.6310679611650486, 'recall': 0.5462184873949579, 'f1': 0.5855855855855856, 'number': 119}	{'precision': 0.8767605633802817, 'recall': 0.924791086350975, 'f1': 0.9001355625847266, 'number': 1077}	0.8614	0.8957	0.8782	0.8110
0.0034	63.1579	1200	1.4398	{'precision': 0.867699642431466, 'recall': 0.8910648714810282, 'f1': 0.8792270531400967, 'number': 817}	{'precision': 0.6288659793814433, 'recall': 0.5126050420168067, 'f1': 0.5648148148148148, 'number': 119}	{'precision': 0.8971533516988063, 'recall': 0.9071494893221913, 'f1': 0.9021237303785781, 'number': 1077}	0.8721	0.8773	0.8747	0.8054
0.0016	73.6842	1400	1.6692	{'precision': 0.8520231213872832, 'recall': 0.9020807833537332, 'f1': 0.8763376932223542, 'number': 817}	{'precision': 0.6039603960396039, 'recall': 0.5126050420168067, 'f1': 0.5545454545454545, 'number': 119}	{'precision': 0.9039923954372624, 'recall': 0.883008356545961, 'f1': 0.8933771723813998, 'number': 1077}	0.8667	0.8689	0.8678	0.7919
0.001	84.2105	1600	1.6412	{'precision': 0.846927374301676, 'recall': 0.9277845777233782, 'f1': 0.8855140186915887, 'number': 817}	{'precision': 0.6095238095238096, 'recall': 0.5378151260504201, 'f1': 0.5714285714285715, 'number': 119}	{'precision': 0.8877828054298642, 'recall': 0.9108635097493036, 'f1': 0.8991750687442712, 'number': 1077}	0.8565	0.8957	0.8757	0.7982
0.0006	94.7368	1800	1.5924	{'precision': 0.8748538011695907, 'recall': 0.9155446756425949, 'f1': 0.8947368421052633, 'number': 817}	{'precision': 0.64, 'recall': 0.5378151260504201, 'f1': 0.5844748858447488, 'number': 119}	{'precision': 0.8945487042001787, 'recall': 0.9294336118848654, 'f1': 0.9116575591985429, 'number': 1077}	0.8742	0.9006	0.8872	0.8193
0.0004	105.2632	2000	1.5639	{'precision': 0.8710433763188745, 'recall': 0.9094247246022031, 'f1': 0.8898203592814371, 'number': 817}	{'precision': 0.6274509803921569, 'recall': 0.5378151260504201, 'f1': 0.579185520361991, 'number': 119}	{'precision': 0.8928892889288929, 'recall': 0.9210770659238626, 'f1': 0.9067641681901281, 'number': 1077}	0.8708	0.8937	0.8821	0.8218
0.0002	115.7895	2200	1.5740	{'precision': 0.8684516880093132, 'recall': 0.9130966952264382, 'f1': 0.8902147971360381, 'number': 817}	{'precision': 0.65, 'recall': 0.5462184873949579, 'f1': 0.593607305936073, 'number': 119}	{'precision': 0.8928247048138056, 'recall': 0.9127205199628597, 'f1': 0.9026629935720845, 'number': 1077}	0.8709	0.8912	0.8809	0.8162
0.0002	126.3158	2400	1.5739	{'precision': 0.8710433763188745, 'recall': 0.9094247246022031, 'f1': 0.8898203592814371, 'number': 817}	{'precision': 0.6336633663366337, 'recall': 0.5378151260504201, 'f1': 0.5818181818181819, 'number': 119}	{'precision': 0.8975521305530372, 'recall': 0.9192200557103064, 'f1': 0.9082568807339448, 'number': 1077}	0.8736	0.8927	0.8830	0.8177

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cpu
• Datasets 3.2.0
• Tokenizers 0.21.0