File size: 7,964 Bytes
b248799 00969a9 b248799 00969a9 b248799 00969a9 b248799 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 |
---
license: mit
base_model: SCUT-DLVCLab/lilt-roberta-en-base
tags:
- generated_from_trainer
model-index:
- name: lilt-en-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# lilt-en-funsd
This model is a fine-tuned version of [SCUT-DLVCLab/lilt-roberta-en-base](https://huggingface.co/SCUT-DLVCLab/lilt-roberta-en-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.8743
- Answer: {'precision': 0.8584579976985041, 'recall': 0.9130966952264382, 'f1': 0.8849347568208777, 'number': 817}
- Header: {'precision': 0.6363636363636364, 'recall': 0.5294117647058824, 'f1': 0.5779816513761468, 'number': 119}
- Question: {'precision': 0.8976014760147601, 'recall': 0.903435468895079, 'f1': 0.9005090236001851, 'number': 1077}
- Overall Precision: 0.8684
- Overall Recall: 0.8852
- Overall F1: 0.8768
- Overall Accuracy: 0.7961
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 2500
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:--------:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 0.0432 | 10.5263 | 200 | 1.2673 | {'precision': 0.8699386503067484, 'recall': 0.8678090575275398, 'f1': 0.8688725490196079, 'number': 817} | {'precision': 0.5602836879432624, 'recall': 0.6638655462184874, 'f1': 0.6076923076923078, 'number': 119} | {'precision': 0.8524734982332155, 'recall': 0.8960074280408542, 'f1': 0.8736985061113626, 'number': 1077} | 0.8396 | 0.8708 | 0.8549 | 0.7871 |
| 0.0111 | 21.0526 | 400 | 1.6035 | {'precision': 0.8281938325991189, 'recall': 0.9204406364749081, 'f1': 0.8718840579710144, 'number': 817} | {'precision': 0.5819672131147541, 'recall': 0.5966386554621849, 'f1': 0.5892116182572614, 'number': 119} | {'precision': 0.8996212121212122, 'recall': 0.8820798514391829, 'f1': 0.8907641819034223, 'number': 1077} | 0.8500 | 0.8808 | 0.8651 | 0.7877 |
| 0.0052 | 31.5789 | 600 | 1.5662 | {'precision': 0.8514619883040936, 'recall': 0.8910648714810282, 'f1': 0.8708133971291867, 'number': 817} | {'precision': 0.6407766990291263, 'recall': 0.5546218487394958, 'f1': 0.5945945945945947, 'number': 119} | {'precision': 0.8783303730017762, 'recall': 0.9182915506035283, 'f1': 0.8978665456196095, 'number': 1077} | 0.8556 | 0.8857 | 0.8704 | 0.7957 |
| 0.0032 | 42.1053 | 800 | 1.7157 | {'precision': 0.8119565217391305, 'recall': 0.9143206854345165, 'f1': 0.8601036269430052, 'number': 817} | {'precision': 0.6867469879518072, 'recall': 0.4789915966386555, 'f1': 0.5643564356435644, 'number': 119} | {'precision': 0.8967136150234741, 'recall': 0.8867223769730733, 'f1': 0.8916900093370681, 'number': 1077} | 0.8506 | 0.8738 | 0.8620 | 0.7843 |
| 0.0023 | 52.6316 | 1000 | 1.7425 | {'precision': 0.8685446009389671, 'recall': 0.9057527539779682, 'f1': 0.8867585380467345, 'number': 817} | {'precision': 0.6283185840707964, 'recall': 0.5966386554621849, 'f1': 0.6120689655172413, 'number': 119} | {'precision': 0.8882931188561215, 'recall': 0.9229340761374187, 'f1': 0.9052823315118397, 'number': 1077} | 0.8661 | 0.8967 | 0.8811 | 0.7941 |
| 0.0011 | 63.1579 | 1200 | 1.8113 | {'precision': 0.8514285714285714, 'recall': 0.9118727050183598, 'f1': 0.8806146572104018, 'number': 817} | {'precision': 0.6272727272727273, 'recall': 0.5798319327731093, 'f1': 0.6026200873362446, 'number': 119} | {'precision': 0.8959854014598541, 'recall': 0.9117920148560817, 'f1': 0.9038196042337783, 'number': 1077} | 0.8630 | 0.8922 | 0.8774 | 0.7946 |
| 0.0005 | 73.6842 | 1400 | 1.8814 | {'precision': 0.8533791523482245, 'recall': 0.9118727050183598, 'f1': 0.8816568047337279, 'number': 817} | {'precision': 0.5362318840579711, 'recall': 0.6218487394957983, 'f1': 0.5758754863813229, 'number': 119} | {'precision': 0.8961646398503275, 'recall': 0.8895078922934077, 'f1': 0.8928238583410998, 'number': 1077} | 0.8543 | 0.8828 | 0.8683 | 0.7877 |
| 0.0006 | 84.2105 | 1600 | 1.9554 | {'precision': 0.8481735159817352, 'recall': 0.9094247246022031, 'f1': 0.8777318369757826, 'number': 817} | {'precision': 0.6052631578947368, 'recall': 0.5798319327731093, 'f1': 0.5922746781115881, 'number': 119} | {'precision': 0.8986301369863013, 'recall': 0.9136490250696379, 'f1': 0.9060773480662985, 'number': 1077} | 0.8614 | 0.8922 | 0.8765 | 0.7946 |
| 0.0002 | 94.7368 | 1800 | 1.9811 | {'precision': 0.8302094818081588, 'recall': 0.9216646266829865, 'f1': 0.8735498839907193, 'number': 817} | {'precision': 0.6464646464646465, 'recall': 0.5378151260504201, 'f1': 0.5871559633027523, 'number': 119} | {'precision': 0.9012115563839702, 'recall': 0.8978644382544104, 'f1': 0.8995348837209303, 'number': 1077} | 0.8581 | 0.8862 | 0.8719 | 0.7903 |
| 0.0004 | 105.2632 | 2000 | 1.8838 | {'precision': 0.8638985005767013, 'recall': 0.9167686658506732, 'f1': 0.8895486935866984, 'number': 817} | {'precision': 0.6146788990825688, 'recall': 0.5630252100840336, 'f1': 0.5877192982456141, 'number': 119} | {'precision': 0.8975069252077562, 'recall': 0.9025069637883009, 'f1': 0.9, 'number': 1077} | 0.8684 | 0.8882 | 0.8782 | 0.7991 |
| 0.0001 | 115.7895 | 2200 | 1.9096 | {'precision': 0.856815578465063, 'recall': 0.9155446756425949, 'f1': 0.8852071005917158, 'number': 817} | {'precision': 0.6203703703703703, 'recall': 0.5630252100840336, 'f1': 0.5903083700440528, 'number': 119} | {'precision': 0.9028944911297853, 'recall': 0.8978644382544104, 'f1': 0.9003724394785848, 'number': 1077} | 0.8684 | 0.8852 | 0.8768 | 0.7952 |
| 0.0001 | 126.3158 | 2400 | 1.8743 | {'precision': 0.8584579976985041, 'recall': 0.9130966952264382, 'f1': 0.8849347568208777, 'number': 817} | {'precision': 0.6363636363636364, 'recall': 0.5294117647058824, 'f1': 0.5779816513761468, 'number': 119} | {'precision': 0.8976014760147601, 'recall': 0.903435468895079, 'f1': 0.9005090236001851, 'number': 1077} | 0.8684 | 0.8852 | 0.8768 | 0.7961 |
### Framework versions
- Transformers 4.44.0
- Pytorch 2.4.0+cu118
- Datasets 2.21.0
- Tokenizers 0.19.1
|