Hugging Face's logo

cdcvd
/
ft-roberta-large-on-bionlp2004-lora

PEFT
Safetensors
Generated from Trainer
Model card Files Community
ft-roberta-large-on-bionlp2004-lora / README.md
cdcvd's picture
cdcvd
End of training
7c55406 verified
preview code
|
raw
history blame
3.84 kB
metadata
base_model: roberta-large
library_name: peft
license: mit
tags:
  - generated_from_trainer
model-index:
  - name: ft-roberta-large-on-bionlp2004-lora
    results: []

ft-roberta-large-on-bionlp2004-lora

This model is a fine-tuned version of roberta-large on the cdcvd/ejpfepj dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0886

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.001
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 50

Training results

Training Loss Epoch Step Validation Loss
No log 1.0 13 0.2459
No log 2.0 26 0.1373
No log 3.0 39 0.1105
No log 4.0 52 0.1414
No log 5.0 65 0.1707
No log 6.0 78 0.1172
No log 7.0 91 0.3309
No log 8.0 104 0.5585
No log 9.0 117 0.5192
No log 10.0 130 0.5445
No log 11.0 143 0.6039
No log 12.0 156 0.5424
No log 13.0 169 0.5210
No log 14.0 182 0.5190
No log 15.0 195 0.5433
No log 16.0 208 0.5199
No log 17.0 221 0.5309
No log 18.0 234 0.5507
No log 19.0 247 0.5427
No log 20.0 260 0.5223
No log 21.0 273 0.5194
No log 22.0 286 0.5216
No log 23.0 299 0.5248
No log 24.0 312 0.5192
No log 25.0 325 0.5409
No log 26.0 338 0.5223
No log 27.0 351 0.5719
No log 28.0 364 0.5307
No log 29.0 377 0.5576
No log 30.0 390 0.5272
No log 31.0 403 0.5193
No log 32.0 416 0.5489
No log 33.0 429 0.5215
No log 34.0 442 0.5359
No log 35.0 455 0.5728
No log 36.0 468 0.5560
No log 37.0 481 0.5765
No log 38.0 494 0.5562
0.4913 39.0 507 0.6608
0.4913 40.0 520 0.7299
0.4913 41.0 533 0.5850
0.4913 42.0 546 0.7992
0.4913 43.0 559 0.7670
0.4913 44.0 572 0.9654
0.4913 45.0 585 1.0347
0.4913 46.0 598 0.9568
0.4913 47.0 611 1.0205
0.4913 48.0 624 1.0679
0.4913 49.0 637 1.1054
0.4913 50.0 650 1.0886

Framework versions

  • PEFT 0.7.1
  • Transformers 4.36.2
  • Pytorch 2.3.1+cu121
  • Datasets 2.15.0
  • Tokenizers 0.15.2