File size: 7,978 Bytes
b248799 13dc855 b248799 00969a9 13dc855 00969a9 b248799 00969a9 b248799 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 |
---
license: mit
base_model: SCUT-DLVCLab/lilt-roberta-en-base
tags:
- generated_from_trainer
model-index:
- name: lilt-en-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# lilt-en-funsd
This model is a fine-tuned version of [SCUT-DLVCLab/lilt-roberta-en-base](https://huggingface.co/SCUT-DLVCLab/lilt-roberta-en-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6142
- Answer: {'precision': 0.8502857142857143, 'recall': 0.9106487148102815, 'f1': 0.8794326241134752, 'number': 817}
- Header: {'precision': 0.638095238095238, 'recall': 0.5630252100840336, 'f1': 0.5982142857142857, 'number': 119}
- Question: {'precision': 0.9045871559633027, 'recall': 0.9155060352831941, 'f1': 0.9100138440239963, 'number': 1077}
- Overall Precision: 0.8681
- Overall Recall: 0.8927
- Overall F1: 0.8802
- Overall Accuracy: 0.8247
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 2500
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:--------:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 0.4043 | 10.5263 | 200 | 0.9334 | {'precision': 0.8331402085747392, 'recall': 0.8800489596083231, 'f1': 0.8559523809523808, 'number': 817} | {'precision': 0.4879518072289157, 'recall': 0.680672268907563, 'f1': 0.5684210526315789, 'number': 119} | {'precision': 0.8621940163191296, 'recall': 0.883008356545961, 'f1': 0.8724770642201833, 'number': 1077} | 0.8213 | 0.8698 | 0.8449 | 0.8028 |
| 0.0458 | 21.0526 | 400 | 1.2878 | {'precision': 0.8611793611793612, 'recall': 0.8580171358629131, 'f1': 0.8595953402820357, 'number': 817} | {'precision': 0.6071428571428571, 'recall': 0.5714285714285714, 'f1': 0.5887445887445888, 'number': 119} | {'precision': 0.8597246127366609, 'recall': 0.9275766016713092, 'f1': 0.892362661902635, 'number': 1077} | 0.8467 | 0.8783 | 0.8622 | 0.8073 |
| 0.0188 | 31.5789 | 600 | 1.2773 | {'precision': 0.8287292817679558, 'recall': 0.9179926560587516, 'f1': 0.8710801393728222, 'number': 817} | {'precision': 0.5892857142857143, 'recall': 0.5546218487394958, 'f1': 0.5714285714285715, 'number': 119} | {'precision': 0.90549662487946, 'recall': 0.871866295264624, 'f1': 0.8883632923368022, 'number': 1077} | 0.8544 | 0.8718 | 0.8630 | 0.7988 |
| 0.007 | 42.1053 | 800 | 1.5029 | {'precision': 0.8728121353558926, 'recall': 0.9155446756425949, 'f1': 0.8936678614097969, 'number': 817} | {'precision': 0.6458333333333334, 'recall': 0.5210084033613446, 'f1': 0.5767441860465117, 'number': 119} | {'precision': 0.8888888888888888, 'recall': 0.9136490250696379, 'f1': 0.9010989010989011, 'number': 1077} | 0.8709 | 0.8912 | 0.8809 | 0.8154 |
| 0.0031 | 52.6316 | 1000 | 1.5006 | {'precision': 0.8540965207631874, 'recall': 0.9314565483476133, 'f1': 0.8911007025761123, 'number': 817} | {'precision': 0.5666666666666667, 'recall': 0.5714285714285714, 'f1': 0.5690376569037656, 'number': 119} | {'precision': 0.9017447199265382, 'recall': 0.9117920148560817, 'f1': 0.9067405355493999, 'number': 1077} | 0.8624 | 0.8997 | 0.8806 | 0.8124 |
| 0.0017 | 63.1579 | 1200 | 1.5541 | {'precision': 0.8778718258766627, 'recall': 0.8886168910648715, 'f1': 0.8832116788321168, 'number': 817} | {'precision': 0.6239316239316239, 'recall': 0.6134453781512605, 'f1': 0.6186440677966102, 'number': 119} | {'precision': 0.8927927927927928, 'recall': 0.9201485608170845, 'f1': 0.9062642889803384, 'number': 1077} | 0.8715 | 0.8892 | 0.8803 | 0.8150 |
| 0.0022 | 73.6842 | 1400 | 1.6132 | {'precision': 0.8556461001164144, 'recall': 0.8996328029375765, 'f1': 0.8770883054892601, 'number': 817} | {'precision': 0.6304347826086957, 'recall': 0.48739495798319327, 'f1': 0.5497630331753555, 'number': 119} | {'precision': 0.8986046511627906, 'recall': 0.8969359331476323, 'f1': 0.8977695167286245, 'number': 1077} | 0.8682 | 0.8738 | 0.8710 | 0.8127 |
| 0.0019 | 84.2105 | 1600 | 1.5373 | {'precision': 0.8615916955017301, 'recall': 0.9143206854345165, 'f1': 0.8871733966745844, 'number': 817} | {'precision': 0.6407766990291263, 'recall': 0.5546218487394958, 'f1': 0.5945945945945947, 'number': 119} | {'precision': 0.8936936936936937, 'recall': 0.9210770659238626, 'f1': 0.9071787837219937, 'number': 1077} | 0.8678 | 0.8967 | 0.8820 | 0.8224 |
| 0.0006 | 94.7368 | 1800 | 1.5759 | {'precision': 0.8616780045351474, 'recall': 0.9302325581395349, 'f1': 0.8946439081812831, 'number': 817} | {'precision': 0.6804123711340206, 'recall': 0.5546218487394958, 'f1': 0.6111111111111112, 'number': 119} | {'precision': 0.9055404178019982, 'recall': 0.9257195914577531, 'f1': 0.9155188246097338, 'number': 1077} | 0.8764 | 0.9056 | 0.8908 | 0.8294 |
| 0.0003 | 105.2632 | 2000 | 1.5537 | {'precision': 0.884004884004884, 'recall': 0.8861689106487148, 'f1': 0.8850855745721272, 'number': 817} | {'precision': 0.6476190476190476, 'recall': 0.5714285714285714, 'f1': 0.6071428571428571, 'number': 119} | {'precision': 0.8874113475177305, 'recall': 0.9294336118848654, 'f1': 0.9079365079365079, 'number': 1077} | 0.8738 | 0.8907 | 0.8822 | 0.8209 |
| 0.0005 | 115.7895 | 2200 | 1.5898 | {'precision': 0.8531791907514451, 'recall': 0.9033047735618115, 'f1': 0.8775267538644471, 'number': 817} | {'precision': 0.591304347826087, 'recall': 0.5714285714285714, 'f1': 0.5811965811965812, 'number': 119} | {'precision': 0.9015496809480401, 'recall': 0.9182915506035283, 'f1': 0.9098436062557498, 'number': 1077} | 0.8642 | 0.8917 | 0.8778 | 0.8223 |
| 0.0002 | 126.3158 | 2400 | 1.6142 | {'precision': 0.8502857142857143, 'recall': 0.9106487148102815, 'f1': 0.8794326241134752, 'number': 817} | {'precision': 0.638095238095238, 'recall': 0.5630252100840336, 'f1': 0.5982142857142857, 'number': 119} | {'precision': 0.9045871559633027, 'recall': 0.9155060352831941, 'f1': 0.9100138440239963, 'number': 1077} | 0.8681 | 0.8927 | 0.8802 | 0.8247 |
### Framework versions
- Transformers 4.44.0
- Pytorch 2.4.0+cu118
- Datasets 2.21.0
- Tokenizers 0.19.1
|