metadata

license: mit
base_model: SCUT-DLVCLab/lilt-roberta-en-base
tags:
  - generated_from_trainer
model-index:
  - name: lilt-en-funsd
    results: []

lilt-en-funsd

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.7177
Answer: {'precision': 0.8729411764705882, 'recall': 0.9082007343941249, 'f1': 0.8902219556088783, 'number': 817}
Header: {'precision': 0.6421052631578947, 'recall': 0.5126050420168067, 'f1': 0.5700934579439252, 'number': 119}
Question: {'precision': 0.906021897810219, 'recall': 0.9220055710306406, 'f1': 0.9139438564196963, 'number': 1077}
Overall Precision: 0.8800
Overall Recall: 0.8922
Overall F1: 0.8860
Overall Accuracy: 0.8064

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
training_steps: 2500
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.4044	10.5263	200	1.1266	{'precision': 0.8246606334841629, 'recall': 0.8922888616891065, 'f1': 0.8571428571428571, 'number': 817}	{'precision': 0.3495575221238938, 'recall': 0.6638655462184874, 'f1': 0.4579710144927537, 'number': 119}	{'precision': 0.8747609942638623, 'recall': 0.8495821727019499, 'f1': 0.8619877531794631, 'number': 1077}	0.7992	0.8559	0.8266	0.7671
0.0518	21.0526	400	1.2142	{'precision': 0.8138006571741512, 'recall': 0.9094247246022031, 'f1': 0.8589595375722543, 'number': 817}	{'precision': 0.49612403100775193, 'recall': 0.5378151260504201, 'f1': 0.5161290322580645, 'number': 119}	{'precision': 0.8705234159779615, 'recall': 0.8802228412256268, 'f1': 0.8753462603878117, 'number': 1077}	0.8236	0.8718	0.8470	0.8011
0.0137	31.5789	600	1.5789	{'precision': 0.8478513356562137, 'recall': 0.8935128518971848, 'f1': 0.8700834326579261, 'number': 817}	{'precision': 0.5588235294117647, 'recall': 0.4789915966386555, 'f1': 0.5158371040723981, 'number': 119}	{'precision': 0.8839528558476881, 'recall': 0.9052924791086351, 'f1': 0.8944954128440367, 'number': 1077}	0.8529	0.8753	0.8639	0.7932
0.008	42.1053	800	1.5466	{'precision': 0.8540478905359179, 'recall': 0.9167686658506732, 'f1': 0.8842975206611571, 'number': 817}	{'precision': 0.528169014084507, 'recall': 0.6302521008403361, 'f1': 0.5747126436781609, 'number': 119}	{'precision': 0.899074074074074, 'recall': 0.9015784586815228, 'f1': 0.9003245248029671, 'number': 1077}	0.8552	0.8917	0.8731	0.7876
0.0047	52.6316	1000	1.5218	{'precision': 0.8712029161603888, 'recall': 0.8776009791921665, 'f1': 0.874390243902439, 'number': 817}	{'precision': 0.5882352941176471, 'recall': 0.5042016806722689, 'f1': 0.5429864253393665, 'number': 119}	{'precision': 0.9080459770114943, 'recall': 0.8802228412256268, 'f1': 0.8939179632248938, 'number': 1077}	0.8761	0.8569	0.8664	0.8023
0.0026	63.1579	1200	1.6588	{'precision': 0.8784596871239471, 'recall': 0.8935128518971848, 'f1': 0.8859223300970873, 'number': 817}	{'precision': 0.532258064516129, 'recall': 0.5546218487394958, 'f1': 0.54320987654321, 'number': 119}	{'precision': 0.8739946380697051, 'recall': 0.9080779944289693, 'f1': 0.8907103825136613, 'number': 1077}	0.8554	0.8813	0.8681	0.7971
0.0013	73.6842	1400	1.6428	{'precision': 0.903822441430333, 'recall': 0.8971848225214198, 'f1': 0.9004914004914006, 'number': 817}	{'precision': 0.6166666666666667, 'recall': 0.6218487394957983, 'f1': 0.6192468619246863, 'number': 119}	{'precision': 0.9017132551848512, 'recall': 0.9285051067780873, 'f1': 0.9149130832570905, 'number': 1077}	0.8858	0.8977	0.8917	0.8127
0.0009	84.2105	1600	1.6516	{'precision': 0.8909090909090909, 'recall': 0.8996328029375765, 'f1': 0.8952496954933008, 'number': 817}	{'precision': 0.6132075471698113, 'recall': 0.5462184873949579, 'f1': 0.5777777777777778, 'number': 119}	{'precision': 0.9070837166513339, 'recall': 0.9155060352831941, 'f1': 0.911275415896488, 'number': 1077}	0.8850	0.8872	0.8861	0.8116
0.0007	94.7368	1800	1.7017	{'precision': 0.8470319634703196, 'recall': 0.9082007343941249, 'f1': 0.8765505020673362, 'number': 817}	{'precision': 0.6521739130434783, 'recall': 0.5042016806722689, 'f1': 0.5687203791469194, 'number': 119}	{'precision': 0.8938547486033519, 'recall': 0.8913649025069638, 'f1': 0.8926080892608089, 'number': 1077}	0.8629	0.8753	0.8691	0.8004
0.0004	105.2632	2000	1.7304	{'precision': 0.8624708624708625, 'recall': 0.9057527539779682, 'f1': 0.8835820895522388, 'number': 817}	{'precision': 0.6095238095238096, 'recall': 0.5378151260504201, 'f1': 0.5714285714285715, 'number': 119}	{'precision': 0.906046511627907, 'recall': 0.904363974001857, 'f1': 0.9052044609665427, 'number': 1077}	0.8724	0.8833	0.8778	0.8019
0.0003	115.7895	2200	1.7230	{'precision': 0.8723404255319149, 'recall': 0.9033047735618115, 'f1': 0.8875526157546603, 'number': 817}	{'precision': 0.6702127659574468, 'recall': 0.5294117647058824, 'f1': 0.5915492957746479, 'number': 119}	{'precision': 0.8992740471869328, 'recall': 0.9201485608170845, 'f1': 0.9095915557595228, 'number': 1077}	0.8776	0.8902	0.8838	0.8049
0.0002	126.3158	2400	1.7177	{'precision': 0.8729411764705882, 'recall': 0.9082007343941249, 'f1': 0.8902219556088783, 'number': 817}	{'precision': 0.6421052631578947, 'recall': 0.5126050420168067, 'f1': 0.5700934579439252, 'number': 119}	{'precision': 0.906021897810219, 'recall': 0.9220055710306406, 'f1': 0.9139438564196963, 'number': 1077}	0.8800	0.8922	0.8860	0.8064

Framework versions

Transformers 4.40.2
Pytorch 2.2.1+cu121
Datasets 2.19.1
Tokenizers 0.19.1