README.md

metadata

license: apache-2.0
tags:
  - generated_from_keras_callback
model-index:
  - name: Regression_bert_1
    results: []

Regression_bert_1

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.1782
• Train Mae: 0.2820
• Train Mse: 0.1314
• Train R2-score: 0.7570
• Validation Loss: 0.1590
• Validation Mae: 0.3493
• Validation Mse: 0.1575
• Validation R2-score: 0.8711
• Epoch: 9

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 1e-04, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Train Mae	Train Mse	Train R2-score	Validation Loss	Validation Mae	Validation Mse	Validation R2-score	Epoch
0.2900	0.2663	0.1055	0.4630	0.1897	0.3738	0.1885	0.8539	0
0.1922	0.3084	0.1536	0.4341	0.2302	0.4254	0.2293	0.8277	1
0.3804	0.3059	0.1401	0.2020	0.1218	0.3206	0.1197	0.8852	2
0.2828	0.3079	0.1426	0.7285	0.1612	0.3507	0.1597	0.8700	3
0.1689	0.2803	0.1282	0.7133	0.2451	0.4425	0.2443	0.8173	4
0.1746	0.2955	0.1469	0.7545	0.2007	0.3887	0.1995	0.8472	5
0.1674	0.2840	0.1372	-4.4884	0.1849	0.3671	0.1836	0.8569	6
0.1691	0.2853	0.1351	0.7348	0.1985	0.3857	0.1973	0.8486	7
0.1615	0.2909	0.1414	0.7511	0.1867	0.3696	0.1854	0.8558	8
0.1782	0.2820	0.1314	0.7570	0.1590	0.3493	0.1575	0.8711	9

Framework versions

• Transformers 4.27.3
• TensorFlow 2.11.0
• Datasets 2.10.1
• Tokenizers 0.13.2