README.md

metadata

library_name: transformers
base_model: moritzbur/lilt-GottBERT-base
tags:
  - generated_from_trainer
datasets:
  - xfund
model-index:
  - name: lilt-GottBERT-base-xfund-de
    results: []

lilt-GottBERT-base-xfund-de

This model is a fine-tuned version of moritzbur/lilt-GottBERT-base on the xfund dataset. It achieves the following results on the evaluation set:

• Loss: 2.0170
• Answer: {'precision': 0.8059836808703535, 'recall': 0.8193548387096774, 'f1': 0.8126142595978061, 'number': 1085}
• Header: {'precision': 0.6590909090909091, 'recall': 0.5, 'f1': 0.5686274509803921, 'number': 58}
• Question: {'precision': 0.7037037037037037, 'recall': 0.837465564738292, 'f1': 0.7647798742138364, 'number': 726}
• Overall Precision: 0.7588
• Overall Recall: 0.8165
• Overall F1: 0.7866
• Overall Accuracy: 0.7433

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.6294	10.5263	200	1.2270	{'precision': 0.7253032928942807, 'recall': 0.7714285714285715, 'f1': 0.7476552032157213, 'number': 1085}	{'precision': 0.4117647058823529, 'recall': 0.4827586206896552, 'f1': 0.4444444444444445, 'number': 58}	{'precision': 0.6296728971962616, 'recall': 0.7424242424242424, 'f1': 0.6814159292035398, 'number': 726}	0.6756	0.7512	0.7114	0.7180
0.0376	21.0526	400	1.5753	{'precision': 0.7734447539461468, 'recall': 0.7677419354838709, 'f1': 0.7705827937095282, 'number': 1085}	{'precision': 0.5, 'recall': 0.39655172413793105, 'f1': 0.4423076923076923, 'number': 58}	{'precision': 0.6336302895322939, 'recall': 0.7837465564738292, 'f1': 0.7007389162561576, 'number': 726}	0.7051	0.7624	0.7326	0.7219
0.0093	31.5789	600	1.7096	{'precision': 0.8057971014492754, 'recall': 0.7686635944700461, 'f1': 0.7867924528301887, 'number': 1085}	{'precision': 0.4807692307692308, 'recall': 0.43103448275862066, 'f1': 0.45454545454545453, 'number': 58}	{'precision': 0.643702906350915, 'recall': 0.8236914600550964, 'f1': 0.7226586102719034, 'number': 726}	0.7227	0.7796	0.7501	0.7347
0.0063	42.1053	800	2.0163	{'precision': 0.760705289672544, 'recall': 0.8350230414746543, 'f1': 0.7961335676625658, 'number': 1085}	{'precision': 0.55, 'recall': 0.3793103448275862, 'f1': 0.4489795918367347, 'number': 58}	{'precision': 0.698090692124105, 'recall': 0.8057851239669421, 'f1': 0.748081841432225, 'number': 726}	0.7313	0.8095	0.7684	0.7047
0.0041	52.6316	1000	1.7820	{'precision': 0.773403324584427, 'recall': 0.8147465437788018, 'f1': 0.793536804308797, 'number': 1085}	{'precision': 0.627906976744186, 'recall': 0.46551724137931033, 'f1': 0.5346534653465347, 'number': 58}	{'precision': 0.6758544652701213, 'recall': 0.8443526170798898, 'f1': 0.7507654623392529, 'number': 726}	0.7281	0.8154	0.7693	0.7344
0.0023	63.1579	1200	1.9631	{'precision': 0.8218390804597702, 'recall': 0.7907834101382488, 'f1': 0.8060122123062471, 'number': 1085}	{'precision': 0.6444444444444445, 'recall': 0.5, 'f1': 0.5631067961165049, 'number': 58}	{'precision': 0.6841491841491841, 'recall': 0.8085399449035813, 'f1': 0.7411616161616162, 'number': 726}	0.7571	0.7887	0.7725	0.7379
0.0016	73.6842	1400	1.8911	{'precision': 0.7939282428702852, 'recall': 0.7953917050691244, 'f1': 0.794659300184162, 'number': 1085}	{'precision': 0.5909090909090909, 'recall': 0.4482758620689655, 'f1': 0.5098039215686274, 'number': 58}	{'precision': 0.6629834254143646, 'recall': 0.8264462809917356, 'f1': 0.7357449417535254, 'number': 726}	0.7313	0.7967	0.7626	0.7442
0.0012	84.2105	1600	1.9599	{'precision': 0.8111954459203036, 'recall': 0.7880184331797235, 'f1': 0.7994389901823281, 'number': 1085}	{'precision': 0.6666666666666666, 'recall': 0.41379310344827586, 'f1': 0.5106382978723404, 'number': 58}	{'precision': 0.7020785219399538, 'recall': 0.837465564738292, 'f1': 0.7638190954773869, 'number': 726}	0.7602	0.7956	0.7775	0.7356
0.0006	94.7368	1800	2.1117	{'precision': 0.8485772357723578, 'recall': 0.7695852534562212, 'f1': 0.8071532141130981, 'number': 1085}	{'precision': 0.6444444444444445, 'recall': 0.5, 'f1': 0.5631067961165049, 'number': 58}	{'precision': 0.7149817295980512, 'recall': 0.8085399449035813, 'f1': 0.7588881706528765, 'number': 726}	0.7843	0.7764	0.7803	0.7377
0.0006	105.2632	2000	2.0033	{'precision': 0.8036866359447005, 'recall': 0.8036866359447005, 'f1': 0.8036866359447006, 'number': 1085}	{'precision': 0.6410256410256411, 'recall': 0.43103448275862066, 'f1': 0.5154639175257731, 'number': 58}	{'precision': 0.6780973451327433, 'recall': 0.8443526170798898, 'f1': 0.7521472392638038, 'number': 726}	0.7446	0.8079	0.7750	0.7409
0.0003	115.7895	2200	2.0170	{'precision': 0.8059836808703535, 'recall': 0.8193548387096774, 'f1': 0.8126142595978061, 'number': 1085}	{'precision': 0.6590909090909091, 'recall': 0.5, 'f1': 0.5686274509803921, 'number': 58}	{'precision': 0.7037037037037037, 'recall': 0.837465564738292, 'f1': 0.7647798742138364, 'number': 726}	0.7588	0.8165	0.7866	0.7433
0.0003	126.3158	2400	2.0173	{'precision': 0.8104761904761905, 'recall': 0.784331797235023, 'f1': 0.7971896955503512, 'number': 1085}	{'precision': 0.6363636363636364, 'recall': 0.4827586206896552, 'f1': 0.5490196078431373, 'number': 58}	{'precision': 0.7096018735362998, 'recall': 0.8347107438016529, 'f1': 0.7670886075949367, 'number': 726}	0.7623	0.7945	0.7781	0.7393

Framework versions

• Transformers 4.45.1
• Pytorch 2.4.0
• Datasets 3.0.1
• Tokenizers 0.20.0