metadata

library_name: transformers
base_model: allenai/scibert_scivocab_uncased
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: my_awesome_model
    results: []

my_awesome_model

This model is a fine-tuned version of allenai/scibert_scivocab_uncased on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.1958
Accuracy: 0.9164

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.1482	0.0507	100	0.2884	0.9044
0.1465	0.1014	200	0.4802	0.8739
0.1523	0.1521	300	0.2684	0.8976
0.1454	0.2027	400	0.4407	0.8696
0.141	0.2534	500	0.2469	0.9106
0.1112	0.3041	600	0.3704	0.8927
0.1368	0.3548	700	0.3799	0.8878
0.0952	0.4055	800	0.3520	0.9041
0.1225	0.4562	900	0.3819	0.8882
0.1245	0.5068	1000	0.2261	0.9101
0.1479	0.5575	1100	0.3054	0.8844
0.1434	0.6082	1200	0.2268	0.9194
0.1622	0.6589	1300	0.2455	0.9053
0.1789	0.7096	1400	0.2411	0.8991
0.1832	0.7603	1500	0.2224	0.9120
0.1855	0.8109	1600	0.2102	0.9105
0.1818	0.8616	1700	0.1893	0.9211
0.1823	0.9123	1800	0.2166	0.9092
0.1632	0.9630	1900	0.1958	0.9164

Framework versions

Transformers 4.44.2
Pytorch 2.4.0
Datasets 3.0.0
Tokenizers 0.19.1