--- license: mit base_model: SCUT-DLVCLab/lilt-roberta-en-base tags: - generated_from_trainer model-index: - name: lilt-en-funsd results: [] --- # lilt-en-funsd This model is a fine-tuned version of [SCUT-DLVCLab/lilt-roberta-en-base](https://huggingface.co/SCUT-DLVCLab/lilt-roberta-en-base) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 1.7648 - Answer: {'precision': 0.8672150411280846, 'recall': 0.9033047735618115, 'f1': 0.8848920863309352, 'number': 817} - Header: {'precision': 0.6666666666666666, 'recall': 0.48739495798319327, 'f1': 0.5631067961165048, 'number': 119} - Question: {'precision': 0.9055912007332723, 'recall': 0.9173630454967502, 'f1': 0.911439114391144, 'number': 1077} - Overall Precision: 0.8793 - Overall Recall: 0.8862 - Overall F1: 0.8827 - Overall Accuracy: 0.8009 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - training_steps: 2500 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |:-------------:|:------:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| | 0.437 | 10.53 | 200 | 0.9767 | {'precision': 0.8161180476730987, 'recall': 0.8800489596083231, 'f1': 0.8468786808009423, 'number': 817} | {'precision': 0.4875, 'recall': 0.3277310924369748, 'f1': 0.39195979899497485, 'number': 119} | {'precision': 0.8519480519480519, 'recall': 0.9136490250696379, 'f1': 0.8817204301075269, 'number': 1077} | 0.8233 | 0.8654 | 0.8438 | 0.7825 | | 0.0411 | 21.05 | 400 | 1.3412 | {'precision': 0.8213507625272332, 'recall': 0.9228886168910648, 'f1': 0.869164265129683, 'number': 817} | {'precision': 0.48936170212765956, 'recall': 0.5798319327731093, 'f1': 0.5307692307692307, 'number': 119} | {'precision': 0.9178217821782179, 'recall': 0.8607242339832869, 'f1': 0.8883564925730715, 'number': 1077} | 0.8458 | 0.8693 | 0.8574 | 0.8058 | | 0.013 | 31.58 | 600 | 1.3818 | {'precision': 0.8340807174887892, 'recall': 0.9106487148102815, 'f1': 0.8706846108835576, 'number': 817} | {'precision': 0.5595238095238095, 'recall': 0.3949579831932773, 'f1': 0.4630541871921182, 'number': 119} | {'precision': 0.8754355400696864, 'recall': 0.9331476323119777, 'f1': 0.903370786516854, 'number': 1077} | 0.8456 | 0.8922 | 0.8683 | 0.8002 | | 0.0079 | 42.11 | 800 | 1.5417 | {'precision': 0.8312849162011173, 'recall': 0.9106487148102815, 'f1': 0.869158878504673, 'number': 817} | {'precision': 0.5789473684210527, 'recall': 0.5546218487394958, 'f1': 0.5665236051502146, 'number': 119} | {'precision': 0.883441258094357, 'recall': 0.8867223769730733, 'f1': 0.8850787766450416, 'number': 1077} | 0.8445 | 0.8768 | 0.8603 | 0.7787 | | 0.0042 | 52.63 | 1000 | 1.7697 | {'precision': 0.8411111111111111, 'recall': 0.9265605875152999, 'f1': 0.881770529994176, 'number': 817} | {'precision': 0.6363636363636364, 'recall': 0.35294117647058826, 'f1': 0.4540540540540541, 'number': 119} | {'precision': 0.8674176776429809, 'recall': 0.9294336118848654, 'f1': 0.897355445988346, 'number': 1077} | 0.8491 | 0.8942 | 0.8710 | 0.7868 | | 0.0025 | 63.16 | 1200 | 1.6700 | {'precision': 0.8520231213872832, 'recall': 0.9020807833537332, 'f1': 0.8763376932223542, 'number': 817} | {'precision': 0.5434782608695652, 'recall': 0.42016806722689076, 'f1': 0.4739336492890995, 'number': 119} | {'precision': 0.8812330009066183, 'recall': 0.9025069637883009, 'f1': 0.891743119266055, 'number': 1077} | 0.8539 | 0.8738 | 0.8637 | 0.7795 | | 0.0013 | 73.68 | 1400 | 1.8217 | {'precision': 0.8444193912063134, 'recall': 0.9167686658506732, 'f1': 0.8791079812206573, 'number': 817} | {'precision': 0.5813953488372093, 'recall': 0.42016806722689076, 'f1': 0.48780487804878053, 'number': 119} | {'precision': 0.8940639269406393, 'recall': 0.9090064995357474, 'f1': 0.9014732965009209, 'number': 1077} | 0.8598 | 0.8833 | 0.8714 | 0.7878 | | 0.0007 | 84.21 | 1600 | 1.7507 | {'precision': 0.8437146092865232, 'recall': 0.9118727050183598, 'f1': 0.8764705882352941, 'number': 817} | {'precision': 0.6794871794871795, 'recall': 0.44537815126050423, 'f1': 0.5380710659898478, 'number': 119} | {'precision': 0.8888888888888888, 'recall': 0.9210770659238626, 'f1': 0.9046967624259006, 'number': 1077} | 0.8618 | 0.8892 | 0.8753 | 0.7901 | | 0.0006 | 94.74 | 1800 | 1.7257 | {'precision': 0.8539976825028969, 'recall': 0.9020807833537332, 'f1': 0.8773809523809523, 'number': 817} | {'precision': 0.6344086021505376, 'recall': 0.4957983193277311, 'f1': 0.5566037735849056, 'number': 119} | {'precision': 0.8943014705882353, 'recall': 0.903435468895079, 'f1': 0.8988452655889145, 'number': 1077} | 0.8655 | 0.8788 | 0.8721 | 0.7922 | | 0.0004 | 105.26 | 2000 | 1.7648 | {'precision': 0.8672150411280846, 'recall': 0.9033047735618115, 'f1': 0.8848920863309352, 'number': 817} | {'precision': 0.6666666666666666, 'recall': 0.48739495798319327, 'f1': 0.5631067961165048, 'number': 119} | {'precision': 0.9055912007332723, 'recall': 0.9173630454967502, 'f1': 0.911439114391144, 'number': 1077} | 0.8793 | 0.8862 | 0.8827 | 0.8009 | | 0.0003 | 115.79 | 2200 | 1.7698 | {'precision': 0.8616279069767442, 'recall': 0.9069767441860465, 'f1': 0.8837209302325582, 'number': 817} | {'precision': 0.6373626373626373, 'recall': 0.48739495798319327, 'f1': 0.5523809523809524, 'number': 119} | {'precision': 0.9104339796860572, 'recall': 0.9155060352831941, 'f1': 0.9129629629629629, 'number': 1077} | 0.8776 | 0.8867 | 0.8821 | 0.8007 | | 0.0003 | 126.32 | 2400 | 1.7623 | {'precision': 0.8596287703016241, 'recall': 0.9069767441860465, 'f1': 0.882668254913639, 'number': 817} | {'precision': 0.5769230769230769, 'recall': 0.5042016806722689, 'f1': 0.5381165919282511, 'number': 119} | {'precision': 0.9018348623853211, 'recall': 0.9127205199628597, 'f1': 0.9072450392247347, 'number': 1077} | 0.8677 | 0.8862 | 0.8769 | 0.7973 | ### Framework versions - Transformers 4.36.2 - Pytorch 2.1.0+cu121 - Datasets 2.16.1 - Tokenizers 0.15.0