metadata

license: mit
base_model: SCUT-DLVCLab/lilt-roberta-en-base
tags:
  - generated_from_trainer
model-index:
  - name: lilt-en-funsd
    results: []

lilt-en-funsd

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.8743
Answer: {'precision': 0.8584579976985041, 'recall': 0.9130966952264382, 'f1': 0.8849347568208777, 'number': 817}
Header: {'precision': 0.6363636363636364, 'recall': 0.5294117647058824, 'f1': 0.5779816513761468, 'number': 119}
Question: {'precision': 0.8976014760147601, 'recall': 0.903435468895079, 'f1': 0.9005090236001851, 'number': 1077}
Overall Precision: 0.8684
Overall Recall: 0.8852
Overall F1: 0.8768
Overall Accuracy: 0.7961

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
training_steps: 2500
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.0432	10.5263	200	1.2673	{'precision': 0.8699386503067484, 'recall': 0.8678090575275398, 'f1': 0.8688725490196079, 'number': 817}	{'precision': 0.5602836879432624, 'recall': 0.6638655462184874, 'f1': 0.6076923076923078, 'number': 119}	{'precision': 0.8524734982332155, 'recall': 0.8960074280408542, 'f1': 0.8736985061113626, 'number': 1077}	0.8396	0.8708	0.8549	0.7871
0.0111	21.0526	400	1.6035	{'precision': 0.8281938325991189, 'recall': 0.9204406364749081, 'f1': 0.8718840579710144, 'number': 817}	{'precision': 0.5819672131147541, 'recall': 0.5966386554621849, 'f1': 0.5892116182572614, 'number': 119}	{'precision': 0.8996212121212122, 'recall': 0.8820798514391829, 'f1': 0.8907641819034223, 'number': 1077}	0.8500	0.8808	0.8651	0.7877
0.0052	31.5789	600	1.5662	{'precision': 0.8514619883040936, 'recall': 0.8910648714810282, 'f1': 0.8708133971291867, 'number': 817}	{'precision': 0.6407766990291263, 'recall': 0.5546218487394958, 'f1': 0.5945945945945947, 'number': 119}	{'precision': 0.8783303730017762, 'recall': 0.9182915506035283, 'f1': 0.8978665456196095, 'number': 1077}	0.8556	0.8857	0.8704	0.7957
0.0032	42.1053	800	1.7157	{'precision': 0.8119565217391305, 'recall': 0.9143206854345165, 'f1': 0.8601036269430052, 'number': 817}	{'precision': 0.6867469879518072, 'recall': 0.4789915966386555, 'f1': 0.5643564356435644, 'number': 119}	{'precision': 0.8967136150234741, 'recall': 0.8867223769730733, 'f1': 0.8916900093370681, 'number': 1077}	0.8506	0.8738	0.8620	0.7843
0.0023	52.6316	1000	1.7425	{'precision': 0.8685446009389671, 'recall': 0.9057527539779682, 'f1': 0.8867585380467345, 'number': 817}	{'precision': 0.6283185840707964, 'recall': 0.5966386554621849, 'f1': 0.6120689655172413, 'number': 119}	{'precision': 0.8882931188561215, 'recall': 0.9229340761374187, 'f1': 0.9052823315118397, 'number': 1077}	0.8661	0.8967	0.8811	0.7941
0.0011	63.1579	1200	1.8113	{'precision': 0.8514285714285714, 'recall': 0.9118727050183598, 'f1': 0.8806146572104018, 'number': 817}	{'precision': 0.6272727272727273, 'recall': 0.5798319327731093, 'f1': 0.6026200873362446, 'number': 119}	{'precision': 0.8959854014598541, 'recall': 0.9117920148560817, 'f1': 0.9038196042337783, 'number': 1077}	0.8630	0.8922	0.8774	0.7946
0.0005	73.6842	1400	1.8814	{'precision': 0.8533791523482245, 'recall': 0.9118727050183598, 'f1': 0.8816568047337279, 'number': 817}	{'precision': 0.5362318840579711, 'recall': 0.6218487394957983, 'f1': 0.5758754863813229, 'number': 119}	{'precision': 0.8961646398503275, 'recall': 0.8895078922934077, 'f1': 0.8928238583410998, 'number': 1077}	0.8543	0.8828	0.8683	0.7877
0.0006	84.2105	1600	1.9554	{'precision': 0.8481735159817352, 'recall': 0.9094247246022031, 'f1': 0.8777318369757826, 'number': 817}	{'precision': 0.6052631578947368, 'recall': 0.5798319327731093, 'f1': 0.5922746781115881, 'number': 119}	{'precision': 0.8986301369863013, 'recall': 0.9136490250696379, 'f1': 0.9060773480662985, 'number': 1077}	0.8614	0.8922	0.8765	0.7946
0.0002	94.7368	1800	1.9811	{'precision': 0.8302094818081588, 'recall': 0.9216646266829865, 'f1': 0.8735498839907193, 'number': 817}	{'precision': 0.6464646464646465, 'recall': 0.5378151260504201, 'f1': 0.5871559633027523, 'number': 119}	{'precision': 0.9012115563839702, 'recall': 0.8978644382544104, 'f1': 0.8995348837209303, 'number': 1077}	0.8581	0.8862	0.8719	0.7903
0.0004	105.2632	2000	1.8838	{'precision': 0.8638985005767013, 'recall': 0.9167686658506732, 'f1': 0.8895486935866984, 'number': 817}	{'precision': 0.6146788990825688, 'recall': 0.5630252100840336, 'f1': 0.5877192982456141, 'number': 119}	{'precision': 0.8975069252077562, 'recall': 0.9025069637883009, 'f1': 0.9, 'number': 1077}	0.8684	0.8882	0.8782	0.7991
0.0001	115.7895	2200	1.9096	{'precision': 0.856815578465063, 'recall': 0.9155446756425949, 'f1': 0.8852071005917158, 'number': 817}	{'precision': 0.6203703703703703, 'recall': 0.5630252100840336, 'f1': 0.5903083700440528, 'number': 119}	{'precision': 0.9028944911297853, 'recall': 0.8978644382544104, 'f1': 0.9003724394785848, 'number': 1077}	0.8684	0.8852	0.8768	0.7952
0.0001	126.3158	2400	1.8743	{'precision': 0.8584579976985041, 'recall': 0.9130966952264382, 'f1': 0.8849347568208777, 'number': 817}	{'precision': 0.6363636363636364, 'recall': 0.5294117647058824, 'f1': 0.5779816513761468, 'number': 119}	{'precision': 0.8976014760147601, 'recall': 0.903435468895079, 'f1': 0.9005090236001851, 'number': 1077}	0.8684	0.8852	0.8768	0.7961

Framework versions

Transformers 4.44.0
Pytorch 2.4.0+cu118
Datasets 2.21.0
Tokenizers 0.19.1