Regression_bert_1

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.1363
• Train Mae: 0.2288
• Train Mse: 0.0928
• Train R2-score: 0.8065
• Validation Loss: 0.0994
• Validation Mae: 0.2297
• Validation Mse: 0.0947
• Validation R2-score: 0.8577
• Epoch: 9

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 1e-04, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Train Mae	Train Mse	Train R2-score	Validation Loss	Validation Mae	Validation Mse	Validation R2-score	Epoch
0.4890	0.3421	0.1734	0.1995	0.1833	0.3662	0.1820	0.8574	0
0.1771	0.3049	0.1526	0.7380	0.1715	0.3553	0.1701	0.8642	1
0.1563	0.2820	0.1337	0.7278	0.1823	0.3662	0.1810	0.8577	2
0.1578	0.2739	0.1288	0.7954	0.1689	0.3535	0.1674	0.8641	3
0.2301	0.3034	0.1383	0.7186	0.1158	0.3132	0.1136	0.8866	4
0.2067	0.2768	0.1203	0.6568	0.1462	0.3375	0.1445	0.8772	5
0.1631	0.2726	0.1200	0.7418	0.1646	0.3473	0.1632	0.8659	6
0.1689	0.2843	0.1269	0.4797	0.0993	0.2797	0.0972	0.8961	7
0.1756	0.2115	0.0726	0.8657	0.1047	0.2537	0.1027	0.8923	8
0.1363	0.2288	0.0928	0.8065	0.0994	0.2297	0.0947	0.8577	9

Framework versions

• Transformers 4.27.3
• TensorFlow 2.11.0
• Datasets 2.10.1
• Tokenizers 0.13.2