|
--- |
|
license: mit |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- funsd-layoutlmv3 |
|
model-index: |
|
- name: new_model |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# new_model |
|
|
|
This model is a fine-tuned version of [SCUT-DLVCLab/lilt-roberta-en-base](https://huggingface.co/SCUT-DLVCLab/lilt-roberta-en-base) on the funsd-layoutlmv3 dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.5095 |
|
- Answer: {'precision': 0.8368479467258602, 'recall': 0.9228886168910648, 'f1': 0.8777648428405123, 'number': 817} |
|
- Header: {'precision': 0.5333333333333333, 'recall': 0.5378151260504201, 'f1': 0.5355648535564853, 'number': 119} |
|
- Question: {'precision': 0.8907407407407407, 'recall': 0.89322191272052, 'f1': 0.8919796012980993, 'number': 1077} |
|
- Overall Precision: 0.8472 |
|
- Overall Recall: 0.8843 |
|
- Overall F1: 0.8653 |
|
- Overall Accuracy: 0.7922 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 5e-05 |
|
- train_batch_size: 4 |
|
- eval_batch_size: 4 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- training_steps: 2500 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| |
|
| 0.1542 | 3.51 | 200 | 0.2369 | {'precision': 0.7753846153846153, 'recall': 0.9253365973072215, 'f1': 0.8437500000000001, 'number': 817} | {'precision': 0.4166666666666667, 'recall': 0.21008403361344538, 'f1': 0.2793296089385475, 'number': 119} | {'precision': 0.8449744463373083, 'recall': 0.9210770659238626, 'f1': 0.8813860506441582, 'number': 1077} | 0.8026 | 0.8808 | 0.8399 | 0.7940 | |
|
| 0.0493 | 7.02 | 400 | 0.2901 | {'precision': 0.8056133056133056, 'recall': 0.9485924112607099, 'f1': 0.8712759977515458, 'number': 817} | {'precision': 0.45614035087719296, 'recall': 0.4369747899159664, 'f1': 0.44635193133047213, 'number': 119} | {'precision': 0.888673765730881, 'recall': 0.8523676880222841, 'f1': 0.8701421800947867, 'number': 1077} | 0.8274 | 0.8669 | 0.8467 | 0.8010 | |
|
| 0.0214 | 10.53 | 600 | 0.3571 | {'precision': 0.8139013452914798, 'recall': 0.8886168910648715, 'f1': 0.8496196606202459, 'number': 817} | {'precision': 0.4186046511627907, 'recall': 0.6050420168067226, 'f1': 0.4948453608247423, 'number': 119} | {'precision': 0.8652946679139383, 'recall': 0.8588672237697307, 'f1': 0.8620689655172413, 'number': 1077} | 0.8078 | 0.8559 | 0.8312 | 0.7907 | |
|
| 0.0112 | 14.04 | 800 | 0.4018 | {'precision': 0.8484455958549223, 'recall': 0.8017135862913096, 'f1': 0.8244178728760226, 'number': 817} | {'precision': 0.45, 'recall': 0.6050420168067226, 'f1': 0.5161290322580645, 'number': 119} | {'precision': 0.8385689354275742, 'recall': 0.8922934076137419, 'f1': 0.8645973909131804, 'number': 1077} | 0.8123 | 0.8385 | 0.8252 | 0.7872 | |
|
| 0.0058 | 17.54 | 1000 | 0.4571 | {'precision': 0.8466666666666667, 'recall': 0.9326805385556916, 'f1': 0.8875946418171229, 'number': 817} | {'precision': 0.6621621621621622, 'recall': 0.4117647058823529, 'f1': 0.5077720207253885, 'number': 119} | {'precision': 0.8809738503155996, 'recall': 0.9071494893221913, 'f1': 0.8938700823421775, 'number': 1077} | 0.8584 | 0.8882 | 0.8730 | 0.7944 | |
|
| 0.0031 | 21.05 | 1200 | 0.4573 | {'precision': 0.8208469055374593, 'recall': 0.9253365973072215, 'f1': 0.8699654775604143, 'number': 817} | {'precision': 0.5727272727272728, 'recall': 0.5294117647058824, 'f1': 0.5502183406113538, 'number': 119} | {'precision': 0.8799266727772685, 'recall': 0.8913649025069638, 'f1': 0.8856088560885608, 'number': 1077} | 0.8384 | 0.8838 | 0.8605 | 0.7891 | |
|
| 0.0019 | 24.56 | 1400 | 0.4631 | {'precision': 0.8372352285395763, 'recall': 0.9192166462668299, 'f1': 0.8763127187864644, 'number': 817} | {'precision': 0.559322033898305, 'recall': 0.5546218487394958, 'f1': 0.5569620253164557, 'number': 119} | {'precision': 0.8864059590316573, 'recall': 0.8839368616527391, 'f1': 0.8851696885169689, 'number': 1077} | 0.8468 | 0.8788 | 0.8625 | 0.7977 | |
|
| 0.0011 | 28.07 | 1600 | 0.5118 | {'precision': 0.838074398249453, 'recall': 0.9375764993880049, 'f1': 0.8850375505488157, 'number': 817} | {'precision': 0.5862068965517241, 'recall': 0.5714285714285714, 'f1': 0.5787234042553192, 'number': 119} | {'precision': 0.885972850678733, 'recall': 0.9090064995357474, 'f1': 0.8973418881759853, 'number': 1077} | 0.8492 | 0.9006 | 0.8742 | 0.7970 | |
|
| 0.0006 | 31.58 | 1800 | 0.4786 | {'precision': 0.8383500557413601, 'recall': 0.9204406364749081, 'f1': 0.8774795799299884, 'number': 817} | {'precision': 0.6, 'recall': 0.5546218487394958, 'f1': 0.5764192139737991, 'number': 119} | {'precision': 0.8851412944393802, 'recall': 0.9015784586815228, 'f1': 0.8932842686292549, 'number': 1077} | 0.8503 | 0.8887 | 0.8691 | 0.7942 | |
|
| 0.0004 | 35.09 | 2000 | 0.4959 | {'precision': 0.8338945005611672, 'recall': 0.9094247246022031, 'f1': 0.870023419203747, 'number': 817} | {'precision': 0.5172413793103449, 'recall': 0.6302521008403361, 'f1': 0.5681818181818182, 'number': 119} | {'precision': 0.8833333333333333, 'recall': 0.8857938718662952, 'f1': 0.8845618915159944, 'number': 1077} | 0.8374 | 0.8803 | 0.8583 | 0.7919 | |
|
| 0.0003 | 38.6 | 2200 | 0.5023 | {'precision': 0.8292682926829268, 'recall': 0.9155446756425949, 'f1': 0.8702734147760327, 'number': 817} | {'precision': 0.5470085470085471, 'recall': 0.5378151260504201, 'f1': 0.5423728813559322, 'number': 119} | {'precision': 0.8771610555050046, 'recall': 0.8950789229340761, 'f1': 0.8860294117647057, 'number': 1077} | 0.8385 | 0.8823 | 0.8598 | 0.7948 | |
|
| 0.0002 | 42.11 | 2400 | 0.5095 | {'precision': 0.8368479467258602, 'recall': 0.9228886168910648, 'f1': 0.8777648428405123, 'number': 817} | {'precision': 0.5333333333333333, 'recall': 0.5378151260504201, 'f1': 0.5355648535564853, 'number': 119} | {'precision': 0.8907407407407407, 'recall': 0.89322191272052, 'f1': 0.8919796012980993, 'number': 1077} | 0.8472 | 0.8843 | 0.8653 | 0.7922 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.28.1 |
|
- Pytorch 2.1.0.dev20230810 |
|
- Datasets 2.14.4 |
|
- Tokenizers 0.11.0 |
|
|
