README.md

metadata

license: mit
base_model: SCUT-DLVCLab/lilt-roberta-en-base
tags:
  - generated_from_trainer
model-index:
  - name: lilt-en-funsd
    results: []

lilt-en-funsd

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.6142
• Answer: {'precision': 0.8502857142857143, 'recall': 0.9106487148102815, 'f1': 0.8794326241134752, 'number': 817}
• Header: {'precision': 0.638095238095238, 'recall': 0.5630252100840336, 'f1': 0.5982142857142857, 'number': 119}
• Question: {'precision': 0.9045871559633027, 'recall': 0.9155060352831941, 'f1': 0.9100138440239963, 'number': 1077}
• Overall Precision: 0.8681
• Overall Recall: 0.8927
• Overall F1: 0.8802
• Overall Accuracy: 0.8247

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.4043	10.5263	200	0.9334	{'precision': 0.8331402085747392, 'recall': 0.8800489596083231, 'f1': 0.8559523809523808, 'number': 817}	{'precision': 0.4879518072289157, 'recall': 0.680672268907563, 'f1': 0.5684210526315789, 'number': 119}	{'precision': 0.8621940163191296, 'recall': 0.883008356545961, 'f1': 0.8724770642201833, 'number': 1077}	0.8213	0.8698	0.8449	0.8028
0.0458	21.0526	400	1.2878	{'precision': 0.8611793611793612, 'recall': 0.8580171358629131, 'f1': 0.8595953402820357, 'number': 817}	{'precision': 0.6071428571428571, 'recall': 0.5714285714285714, 'f1': 0.5887445887445888, 'number': 119}	{'precision': 0.8597246127366609, 'recall': 0.9275766016713092, 'f1': 0.892362661902635, 'number': 1077}	0.8467	0.8783	0.8622	0.8073
0.0188	31.5789	600	1.2773	{'precision': 0.8287292817679558, 'recall': 0.9179926560587516, 'f1': 0.8710801393728222, 'number': 817}	{'precision': 0.5892857142857143, 'recall': 0.5546218487394958, 'f1': 0.5714285714285715, 'number': 119}	{'precision': 0.90549662487946, 'recall': 0.871866295264624, 'f1': 0.8883632923368022, 'number': 1077}	0.8544	0.8718	0.8630	0.7988
0.007	42.1053	800	1.5029	{'precision': 0.8728121353558926, 'recall': 0.9155446756425949, 'f1': 0.8936678614097969, 'number': 817}	{'precision': 0.6458333333333334, 'recall': 0.5210084033613446, 'f1': 0.5767441860465117, 'number': 119}	{'precision': 0.8888888888888888, 'recall': 0.9136490250696379, 'f1': 0.9010989010989011, 'number': 1077}	0.8709	0.8912	0.8809	0.8154
0.0031	52.6316	1000	1.5006	{'precision': 0.8540965207631874, 'recall': 0.9314565483476133, 'f1': 0.8911007025761123, 'number': 817}	{'precision': 0.5666666666666667, 'recall': 0.5714285714285714, 'f1': 0.5690376569037656, 'number': 119}	{'precision': 0.9017447199265382, 'recall': 0.9117920148560817, 'f1': 0.9067405355493999, 'number': 1077}	0.8624	0.8997	0.8806	0.8124
0.0017	63.1579	1200	1.5541	{'precision': 0.8778718258766627, 'recall': 0.8886168910648715, 'f1': 0.8832116788321168, 'number': 817}	{'precision': 0.6239316239316239, 'recall': 0.6134453781512605, 'f1': 0.6186440677966102, 'number': 119}	{'precision': 0.8927927927927928, 'recall': 0.9201485608170845, 'f1': 0.9062642889803384, 'number': 1077}	0.8715	0.8892	0.8803	0.8150
0.0022	73.6842	1400	1.6132	{'precision': 0.8556461001164144, 'recall': 0.8996328029375765, 'f1': 0.8770883054892601, 'number': 817}	{'precision': 0.6304347826086957, 'recall': 0.48739495798319327, 'f1': 0.5497630331753555, 'number': 119}	{'precision': 0.8986046511627906, 'recall': 0.8969359331476323, 'f1': 0.8977695167286245, 'number': 1077}	0.8682	0.8738	0.8710	0.8127
0.0019	84.2105	1600	1.5373	{'precision': 0.8615916955017301, 'recall': 0.9143206854345165, 'f1': 0.8871733966745844, 'number': 817}	{'precision': 0.6407766990291263, 'recall': 0.5546218487394958, 'f1': 0.5945945945945947, 'number': 119}	{'precision': 0.8936936936936937, 'recall': 0.9210770659238626, 'f1': 0.9071787837219937, 'number': 1077}	0.8678	0.8967	0.8820	0.8224
0.0006	94.7368	1800	1.5759	{'precision': 0.8616780045351474, 'recall': 0.9302325581395349, 'f1': 0.8946439081812831, 'number': 817}	{'precision': 0.6804123711340206, 'recall': 0.5546218487394958, 'f1': 0.6111111111111112, 'number': 119}	{'precision': 0.9055404178019982, 'recall': 0.9257195914577531, 'f1': 0.9155188246097338, 'number': 1077}	0.8764	0.9056	0.8908	0.8294
0.0003	105.2632	2000	1.5537	{'precision': 0.884004884004884, 'recall': 0.8861689106487148, 'f1': 0.8850855745721272, 'number': 817}	{'precision': 0.6476190476190476, 'recall': 0.5714285714285714, 'f1': 0.6071428571428571, 'number': 119}	{'precision': 0.8874113475177305, 'recall': 0.9294336118848654, 'f1': 0.9079365079365079, 'number': 1077}	0.8738	0.8907	0.8822	0.8209
0.0005	115.7895	2200	1.5898	{'precision': 0.8531791907514451, 'recall': 0.9033047735618115, 'f1': 0.8775267538644471, 'number': 817}	{'precision': 0.591304347826087, 'recall': 0.5714285714285714, 'f1': 0.5811965811965812, 'number': 119}	{'precision': 0.9015496809480401, 'recall': 0.9182915506035283, 'f1': 0.9098436062557498, 'number': 1077}	0.8642	0.8917	0.8778	0.8223
0.0002	126.3158	2400	1.6142	{'precision': 0.8502857142857143, 'recall': 0.9106487148102815, 'f1': 0.8794326241134752, 'number': 817}	{'precision': 0.638095238095238, 'recall': 0.5630252100840336, 'f1': 0.5982142857142857, 'number': 119}	{'precision': 0.9045871559633027, 'recall': 0.9155060352831941, 'f1': 0.9100138440239963, 'number': 1077}	0.8681	0.8927	0.8802	0.8247

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0+cu118
• Datasets 2.21.0
• Tokenizers 0.19.1