kasrahabib/roberta-base-finetuned-iso29148-init-promise-all-classes-classifier

This model is a fine-tuned version of FacebookAI/roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.2095
• Validation Loss: 1.0744
• Epoch: 29

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 480, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
2.4272	2.3997	0
2.4041	2.3823	1
2.3060	2.2115	2
1.8349	1.7804	3
1.1829	1.3933	4
0.7792	1.2214	5
0.5448	1.1223	6
0.3766	1.1217	7
0.3057	1.1044	8
0.2453	1.0735	9
0.2230	1.0699	10
0.2079	1.0752	11
0.2115	1.0744	12
0.2084	1.0744	13
0.2032	1.0744	14
0.2044	1.0744	15
0.2115	1.0744	16
0.2091	1.0744	17
0.2030	1.0744	18
0.2137	1.0744	19
0.2056	1.0744	20
0.2057	1.0744	21
0.2099	1.0744	22
0.1919	1.0744	23
0.2152	1.0744	24
0.2013	1.0744	25
0.2072	1.0744	26
0.2044	1.0744	27
0.2050	1.0744	28
0.2095	1.0744	29

Framework versions

• Transformers 4.42.3
• TensorFlow 2.15.0
• Datasets 2.19.1
• Tokenizers 0.19.1