metadata

license: mit
tags:
  - generated_from_trainer
datasets:
  - funsd-layoutlmv3
model-index:
  - name: new_model
    results: []

new_model

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on the funsd-layoutlmv3 dataset. It achieves the following results on the evaluation set:

Loss: 0.5095
Answer: {'precision': 0.8368479467258602, 'recall': 0.9228886168910648, 'f1': 0.8777648428405123, 'number': 817}
Header: {'precision': 0.5333333333333333, 'recall': 0.5378151260504201, 'f1': 0.5355648535564853, 'number': 119}
Question: {'precision': 0.8907407407407407, 'recall': 0.89322191272052, 'f1': 0.8919796012980993, 'number': 1077}
Overall Precision: 0.8472
Overall Recall: 0.8843
Overall F1: 0.8653
Overall Accuracy: 0.7922

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
training_steps: 2500

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.1542	3.51	200	0.2369	{'precision': 0.7753846153846153, 'recall': 0.9253365973072215, 'f1': 0.8437500000000001, 'number': 817}	{'precision': 0.4166666666666667, 'recall': 0.21008403361344538, 'f1': 0.2793296089385475, 'number': 119}	{'precision': 0.8449744463373083, 'recall': 0.9210770659238626, 'f1': 0.8813860506441582, 'number': 1077}	0.8026	0.8808	0.8399	0.7940
0.0493	7.02	400	0.2901	{'precision': 0.8056133056133056, 'recall': 0.9485924112607099, 'f1': 0.8712759977515458, 'number': 817}	{'precision': 0.45614035087719296, 'recall': 0.4369747899159664, 'f1': 0.44635193133047213, 'number': 119}	{'precision': 0.888673765730881, 'recall': 0.8523676880222841, 'f1': 0.8701421800947867, 'number': 1077}	0.8274	0.8669	0.8467	0.8010
0.0214	10.53	600	0.3571	{'precision': 0.8139013452914798, 'recall': 0.8886168910648715, 'f1': 0.8496196606202459, 'number': 817}	{'precision': 0.4186046511627907, 'recall': 0.6050420168067226, 'f1': 0.4948453608247423, 'number': 119}	{'precision': 0.8652946679139383, 'recall': 0.8588672237697307, 'f1': 0.8620689655172413, 'number': 1077}	0.8078	0.8559	0.8312	0.7907
0.0112	14.04	800	0.4018	{'precision': 0.8484455958549223, 'recall': 0.8017135862913096, 'f1': 0.8244178728760226, 'number': 817}	{'precision': 0.45, 'recall': 0.6050420168067226, 'f1': 0.5161290322580645, 'number': 119}	{'precision': 0.8385689354275742, 'recall': 0.8922934076137419, 'f1': 0.8645973909131804, 'number': 1077}	0.8123	0.8385	0.8252	0.7872
0.0058	17.54	1000	0.4571	{'precision': 0.8466666666666667, 'recall': 0.9326805385556916, 'f1': 0.8875946418171229, 'number': 817}	{'precision': 0.6621621621621622, 'recall': 0.4117647058823529, 'f1': 0.5077720207253885, 'number': 119}	{'precision': 0.8809738503155996, 'recall': 0.9071494893221913, 'f1': 0.8938700823421775, 'number': 1077}	0.8584	0.8882	0.8730	0.7944
0.0031	21.05	1200	0.4573	{'precision': 0.8208469055374593, 'recall': 0.9253365973072215, 'f1': 0.8699654775604143, 'number': 817}	{'precision': 0.5727272727272728, 'recall': 0.5294117647058824, 'f1': 0.5502183406113538, 'number': 119}	{'precision': 0.8799266727772685, 'recall': 0.8913649025069638, 'f1': 0.8856088560885608, 'number': 1077}	0.8384	0.8838	0.8605	0.7891
0.0019	24.56	1400	0.4631	{'precision': 0.8372352285395763, 'recall': 0.9192166462668299, 'f1': 0.8763127187864644, 'number': 817}	{'precision': 0.559322033898305, 'recall': 0.5546218487394958, 'f1': 0.5569620253164557, 'number': 119}	{'precision': 0.8864059590316573, 'recall': 0.8839368616527391, 'f1': 0.8851696885169689, 'number': 1077}	0.8468	0.8788	0.8625	0.7977
0.0011	28.07	1600	0.5118	{'precision': 0.838074398249453, 'recall': 0.9375764993880049, 'f1': 0.8850375505488157, 'number': 817}	{'precision': 0.5862068965517241, 'recall': 0.5714285714285714, 'f1': 0.5787234042553192, 'number': 119}	{'precision': 0.885972850678733, 'recall': 0.9090064995357474, 'f1': 0.8973418881759853, 'number': 1077}	0.8492	0.9006	0.8742	0.7970
0.0006	31.58	1800	0.4786	{'precision': 0.8383500557413601, 'recall': 0.9204406364749081, 'f1': 0.8774795799299884, 'number': 817}	{'precision': 0.6, 'recall': 0.5546218487394958, 'f1': 0.5764192139737991, 'number': 119}	{'precision': 0.8851412944393802, 'recall': 0.9015784586815228, 'f1': 0.8932842686292549, 'number': 1077}	0.8503	0.8887	0.8691	0.7942
0.0004	35.09	2000	0.4959	{'precision': 0.8338945005611672, 'recall': 0.9094247246022031, 'f1': 0.870023419203747, 'number': 817}	{'precision': 0.5172413793103449, 'recall': 0.6302521008403361, 'f1': 0.5681818181818182, 'number': 119}	{'precision': 0.8833333333333333, 'recall': 0.8857938718662952, 'f1': 0.8845618915159944, 'number': 1077}	0.8374	0.8803	0.8583	0.7919
0.0003	38.6	2200	0.5023	{'precision': 0.8292682926829268, 'recall': 0.9155446756425949, 'f1': 0.8702734147760327, 'number': 817}	{'precision': 0.5470085470085471, 'recall': 0.5378151260504201, 'f1': 0.5423728813559322, 'number': 119}	{'precision': 0.8771610555050046, 'recall': 0.8950789229340761, 'f1': 0.8860294117647057, 'number': 1077}	0.8385	0.8823	0.8598	0.7948
0.0002	42.11	2400	0.5095	{'precision': 0.8368479467258602, 'recall': 0.9228886168910648, 'f1': 0.8777648428405123, 'number': 817}	{'precision': 0.5333333333333333, 'recall': 0.5378151260504201, 'f1': 0.5355648535564853, 'number': 119}	{'precision': 0.8907407407407407, 'recall': 0.89322191272052, 'f1': 0.8919796012980993, 'number': 1077}	0.8472	0.8843	0.8653	0.7922

Framework versions

Transformers 4.28.1
Pytorch 2.1.0.dev20230810
Datasets 2.14.4
Tokenizers 0.11.0