Hugging Face's logo

Edwinlasso99
/
xml-roberta-large-32size

PEFT
TensorBoard
Safetensors
Generated from Trainer
Model card Files Metrics Community
Edwinlasso99 commited on
Commit
06246bb
·
verified ·
1 Parent(s): cecf70e

End of training

Browse files
Files changed (1) hide show
  1. README.md +146 -0
README.md ADDED
@@ -0,0 +1,146 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ license: mit
4
+ base_model: FacebookAI/xlm-roberta-large
5
+ tags:
6
+ - generated_from_trainer
7
+ datasets:
8
+ - biobert_json
9
+ metrics:
10
+ - precision
11
+ - recall
12
+ - f1
13
+ - accuracy
14
+ model-index:
15
+ - name: xml-roberta-large-32size
16
+ results: []
17
+ ---
18
+
19
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
20
+ should probably proofread and complete it, then remove this comment. -->
21
+
22
+ # xml-roberta-large-32size
23
+
24
+ This model is a fine-tuned version of [FacebookAI/xlm-roberta-large](https://huggingface.co/FacebookAI/xlm-roberta-large) on the biobert_json dataset.
25
+ It achieves the following results on the evaluation set:
26
+ - Loss: 0.0713
27
+ - Precision: 0.9384
28
+ - Recall: 0.9606
29
+ - F1: 0.9494
30
+ - Accuracy: 0.9815
31
+
32
+ ## Model description
33
+
34
+ More information needed
35
+
36
+ ## Intended uses & limitations
37
+
38
+ More information needed
39
+
40
+ ## Training and evaluation data
41
+
42
+ More information needed
43
+
44
+ ## Training procedure
45
+
46
+ ### Training hyperparameters
47
+
48
+ The following hyperparameters were used during training:
49
+ - learning_rate: 0.0004
50
+ - train_batch_size: 32
51
+ - eval_batch_size: 32
52
+ - seed: 42
53
+ - optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
54
+ - lr_scheduler_type: linear
55
+ - training_steps: 1525
56
+ - mixed_precision_training: Native AMP
57
+
58
+ ### Training results
59
+
60
+ | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
61
+ |:-------------:|:------:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
62
+ | 2.1444 | 0.0654 | 20 | 1.0529 | 0.0018 | 0.0001 | 0.0002 | 0.7296 |
63
+ | 0.7846 | 0.1307 | 40 | 0.4058 | 0.8486 | 0.6892 | 0.7606 | 0.9006 |
64
+ | 0.4425 | 0.1961 | 60 | 0.2294 | 0.8464 | 0.8005 | 0.8228 | 0.9367 |
65
+ | 0.2825 | 0.2614 | 80 | 0.1704 | 0.8433 | 0.8671 | 0.8551 | 0.9507 |
66
+ | 0.2151 | 0.3268 | 100 | 0.1418 | 0.8516 | 0.9131 | 0.8813 | 0.9595 |
67
+ | 0.2432 | 0.3922 | 120 | 0.1401 | 0.8586 | 0.9094 | 0.8833 | 0.9604 |
68
+ | 0.1772 | 0.4575 | 140 | 0.1383 | 0.9107 | 0.8861 | 0.8982 | 0.9632 |
69
+ | 0.1812 | 0.5229 | 160 | 0.1275 | 0.8763 | 0.9028 | 0.8893 | 0.9619 |
70
+ | 0.1687 | 0.5882 | 180 | 0.1468 | 0.8571 | 0.9317 | 0.8928 | 0.9586 |
71
+ | 0.1434 | 0.6536 | 200 | 0.1088 | 0.8913 | 0.9301 | 0.9103 | 0.9684 |
72
+ | 0.1415 | 0.7190 | 220 | 0.1022 | 0.9059 | 0.9093 | 0.9076 | 0.9684 |
73
+ | 0.1444 | 0.7843 | 240 | 0.1205 | 0.8873 | 0.9351 | 0.9106 | 0.9674 |
74
+ | 0.1306 | 0.8497 | 260 | 0.1012 | 0.9145 | 0.9194 | 0.9170 | 0.9697 |
75
+ | 0.1389 | 0.9150 | 280 | 0.1152 | 0.8834 | 0.9329 | 0.9074 | 0.9653 |
76
+ | 0.1403 | 0.9804 | 300 | 0.0920 | 0.9120 | 0.9325 | 0.9221 | 0.9728 |
77
+ | 0.111 | 1.0458 | 320 | 0.0930 | 0.9091 | 0.9373 | 0.9230 | 0.9728 |
78
+ | 0.1031 | 1.1111 | 340 | 0.0987 | 0.9024 | 0.9420 | 0.9218 | 0.9707 |
79
+ | 0.1123 | 1.1765 | 360 | 0.0897 | 0.9193 | 0.9266 | 0.9229 | 0.9726 |
80
+ | 0.1131 | 1.2418 | 380 | 0.0845 | 0.9260 | 0.9332 | 0.9296 | 0.9761 |
81
+ | 0.1105 | 1.3072 | 400 | 0.0808 | 0.9206 | 0.9442 | 0.9323 | 0.9764 |
82
+ | 0.1072 | 1.3725 | 420 | 0.0923 | 0.8897 | 0.9277 | 0.9083 | 0.9710 |
83
+ | 0.1018 | 1.4379 | 440 | 0.0777 | 0.9254 | 0.9458 | 0.9355 | 0.9779 |
84
+ | 0.1004 | 1.5033 | 460 | 0.0941 | 0.9142 | 0.9485 | 0.9310 | 0.9747 |
85
+ | 0.0932 | 1.5686 | 480 | 0.0827 | 0.9191 | 0.9549 | 0.9367 | 0.9768 |
86
+ | 0.0859 | 1.6340 | 500 | 0.0897 | 0.9165 | 0.9512 | 0.9335 | 0.9749 |
87
+ | 0.0707 | 1.6993 | 520 | 0.0875 | 0.9093 | 0.9616 | 0.9347 | 0.9743 |
88
+ | 0.0851 | 1.7647 | 540 | 0.0764 | 0.9254 | 0.9472 | 0.9362 | 0.9781 |
89
+ | 0.0989 | 1.8301 | 560 | 0.0928 | 0.9044 | 0.9491 | 0.9262 | 0.9719 |
90
+ | 0.0909 | 1.8954 | 580 | 0.1008 | 0.8954 | 0.9555 | 0.9244 | 0.9712 |
91
+ | 0.0865 | 1.9608 | 600 | 0.0887 | 0.9067 | 0.9508 | 0.9283 | 0.9741 |
92
+ | 0.077 | 2.0261 | 620 | 0.0831 | 0.9140 | 0.9573 | 0.9351 | 0.9763 |
93
+ | 0.0581 | 2.0915 | 640 | 0.0731 | 0.9305 | 0.9537 | 0.9420 | 0.9795 |
94
+ | 0.052 | 2.1569 | 660 | 0.0750 | 0.9165 | 0.9449 | 0.9305 | 0.9770 |
95
+ | 0.0781 | 2.2222 | 680 | 0.0744 | 0.9337 | 0.9531 | 0.9433 | 0.9787 |
96
+ | 0.0738 | 2.2876 | 700 | 0.0766 | 0.9260 | 0.9528 | 0.9392 | 0.9790 |
97
+ | 0.0587 | 2.3529 | 720 | 0.0680 | 0.9383 | 0.9534 | 0.9458 | 0.9806 |
98
+ | 0.0641 | 2.4183 | 740 | 0.0690 | 0.9330 | 0.9544 | 0.9436 | 0.9799 |
99
+ | 0.0837 | 2.4837 | 760 | 0.0687 | 0.9342 | 0.9589 | 0.9464 | 0.9812 |
100
+ | 0.0673 | 2.5490 | 780 | 0.0728 | 0.9383 | 0.9607 | 0.9494 | 0.9814 |
101
+ | 0.0687 | 2.6144 | 800 | 0.0715 | 0.9403 | 0.9570 | 0.9486 | 0.9813 |
102
+ | 0.0594 | 2.6797 | 820 | 0.0758 | 0.9322 | 0.9499 | 0.9410 | 0.9783 |
103
+ | 0.0726 | 2.7451 | 840 | 0.0733 | 0.9297 | 0.9549 | 0.9421 | 0.9798 |
104
+ | 0.0596 | 2.8105 | 860 | 0.0733 | 0.9343 | 0.9600 | 0.9470 | 0.9802 |
105
+ | 0.0645 | 2.8758 | 880 | 0.0745 | 0.9336 | 0.9585 | 0.9458 | 0.9791 |
106
+ | 0.0544 | 2.9412 | 900 | 0.0728 | 0.9300 | 0.9600 | 0.9448 | 0.9797 |
107
+ | 0.0518 | 3.0065 | 920 | 0.0672 | 0.9412 | 0.9610 | 0.9510 | 0.9821 |
108
+ | 0.0444 | 3.0719 | 940 | 0.0719 | 0.9340 | 0.9587 | 0.9462 | 0.9807 |
109
+ | 0.0518 | 3.1373 | 960 | 0.0810 | 0.9266 | 0.9579 | 0.9420 | 0.9783 |
110
+ | 0.0448 | 3.2026 | 980 | 0.0723 | 0.9365 | 0.9577 | 0.9470 | 0.9810 |
111
+ | 0.0546 | 3.2680 | 1000 | 0.0775 | 0.9272 | 0.9604 | 0.9435 | 0.9782 |
112
+ | 0.0516 | 3.3333 | 1020 | 0.0725 | 0.9310 | 0.9508 | 0.9408 | 0.9794 |
113
+ | 0.0513 | 3.3987 | 1040 | 0.0751 | 0.9295 | 0.9656 | 0.9472 | 0.9804 |
114
+ | 0.0567 | 3.4641 | 1060 | 0.0694 | 0.9348 | 0.9579 | 0.9462 | 0.9807 |
115
+ | 0.046 | 3.5294 | 1080 | 0.0688 | 0.9306 | 0.9566 | 0.9434 | 0.9803 |
116
+ | 0.0476 | 3.5948 | 1100 | 0.0825 | 0.9189 | 0.9532 | 0.9357 | 0.9764 |
117
+ | 0.0399 | 3.6601 | 1120 | 0.0700 | 0.9390 | 0.9568 | 0.9478 | 0.9816 |
118
+ | 0.0629 | 3.7255 | 1140 | 0.0680 | 0.9398 | 0.9586 | 0.9491 | 0.9813 |
119
+ | 0.0509 | 3.7908 | 1160 | 0.0713 | 0.9379 | 0.9598 | 0.9487 | 0.9812 |
120
+ | 0.0578 | 3.8562 | 1180 | 0.0738 | 0.9330 | 0.9615 | 0.9470 | 0.9801 |
121
+ | 0.0565 | 3.9216 | 1200 | 0.0826 | 0.9288 | 0.9603 | 0.9443 | 0.9776 |
122
+ | 0.0603 | 3.9869 | 1220 | 0.0678 | 0.9417 | 0.9589 | 0.9502 | 0.9819 |
123
+ | 0.0429 | 4.0523 | 1240 | 0.0734 | 0.9299 | 0.9597 | 0.9446 | 0.9802 |
124
+ | 0.0338 | 4.1176 | 1260 | 0.0719 | 0.9321 | 0.9564 | 0.9441 | 0.9803 |
125
+ | 0.0397 | 4.1830 | 1280 | 0.0688 | 0.9411 | 0.9616 | 0.9512 | 0.9827 |
126
+ | 0.0432 | 4.2484 | 1300 | 0.0719 | 0.9370 | 0.9588 | 0.9478 | 0.9814 |
127
+ | 0.0403 | 4.3137 | 1320 | 0.0697 | 0.9423 | 0.9557 | 0.9490 | 0.9819 |
128
+ | 0.0418 | 4.3791 | 1340 | 0.0706 | 0.9409 | 0.9619 | 0.9513 | 0.9822 |
129
+ | 0.0345 | 4.4444 | 1360 | 0.0704 | 0.9408 | 0.9617 | 0.9511 | 0.9821 |
130
+ | 0.0297 | 4.5098 | 1380 | 0.0734 | 0.9356 | 0.9629 | 0.9490 | 0.9814 |
131
+ | 0.0424 | 4.5752 | 1400 | 0.0723 | 0.9376 | 0.9601 | 0.9487 | 0.9816 |
132
+ | 0.0421 | 4.6405 | 1420 | 0.0708 | 0.9388 | 0.9594 | 0.9490 | 0.9819 |
133
+ | 0.0343 | 4.7059 | 1440 | 0.0736 | 0.9337 | 0.9600 | 0.9467 | 0.9810 |
134
+ | 0.0432 | 4.7712 | 1460 | 0.0730 | 0.9356 | 0.9604 | 0.9478 | 0.9812 |
135
+ | 0.0401 | 4.8366 | 1480 | 0.0723 | 0.9377 | 0.9615 | 0.9494 | 0.9814 |
136
+ | 0.0382 | 4.9020 | 1500 | 0.0714 | 0.9387 | 0.9609 | 0.9497 | 0.9815 |
137
+ | 0.044 | 4.9673 | 1520 | 0.0713 | 0.9384 | 0.9606 | 0.9494 | 0.9815 |
138
+
139
+
140
+ ### Framework versions
141
+
142
+ - PEFT 0.13.2
143
+ - Transformers 4.47.0
144
+ - Pytorch 2.5.1+cu121
145
+ - Datasets 3.2.0
146
+ - Tokenizers 0.21.0