Hugging Face's logo

brandonRivas
/
xml-roberta-large-ner-qlorafinetune-runs-colab-32size

PEFT
TensorBoard
Safetensors
Generated from Trainer
Model card Files Metrics Community
brandonRivas commited on
Commit
38ef362
·
verified ·
1 Parent(s): b85ee4d

End of training

Browse files
Files changed (1) hide show
  1. README.md +177 -0
README.md ADDED
@@ -0,0 +1,177 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ license: mit
4
+ base_model: FacebookAI/xlm-roberta-large
5
+ tags:
6
+ - generated_from_trainer
7
+ datasets:
8
+ - biobert_json
9
+ metrics:
10
+ - precision
11
+ - recall
12
+ - f1
13
+ - accuracy
14
+ model-index:
15
+ - name: xml-roberta-large-ner-qlorafinetune-runs-colab-32size
16
+ results: []
17
+ ---
18
+
19
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
20
+ should probably proofread and complete it, then remove this comment. -->
21
+
22
+ # xml-roberta-large-ner-qlorafinetune-runs-colab-32size
23
+
24
+ This model is a fine-tuned version of [FacebookAI/xlm-roberta-large](https://huggingface.co/FacebookAI/xlm-roberta-large) on the biobert_json dataset.
25
+ It achieves the following results on the evaluation set:
26
+ - Loss: 0.0784
27
+ - Precision: 0.9381
28
+ - Recall: 0.9575
29
+ - F1: 0.9477
30
+ - Accuracy: 0.9813
31
+
32
+ ## Model description
33
+
34
+ More information needed
35
+
36
+ ## Intended uses & limitations
37
+
38
+ More information needed
39
+
40
+ ## Training and evaluation data
41
+
42
+ More information needed
43
+
44
+ ## Training procedure
45
+
46
+ ### Training hyperparameters
47
+
48
+ The following hyperparameters were used during training:
49
+ - learning_rate: 0.0004
50
+ - train_batch_size: 32
51
+ - eval_batch_size: 32
52
+ - seed: 42
53
+ - optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
54
+ - lr_scheduler_type: linear
55
+ - training_steps: 2141
56
+ - mixed_precision_training: Native AMP
57
+
58
+ ### Training results
59
+
60
+ | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
61
+ |:-------------:|:------:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
62
+ | 2.3478 | 0.0654 | 20 | 1.3438 | 0.0 | 0.0 | 0.0 | 0.7180 |
63
+ | 1.2529 | 0.1307 | 40 | 0.8221 | 0.4886 | 0.4719 | 0.4801 | 0.8204 |
64
+ | 0.741 | 0.1961 | 60 | 0.4377 | 0.7634 | 0.6193 | 0.6838 | 0.8838 |
65
+ | 0.4179 | 0.2614 | 80 | 0.2434 | 0.8013 | 0.7903 | 0.7958 | 0.9309 |
66
+ | 0.301 | 0.3268 | 100 | 0.1965 | 0.8393 | 0.8788 | 0.8586 | 0.9477 |
67
+ | 0.2704 | 0.3922 | 120 | 0.1662 | 0.8354 | 0.8823 | 0.8583 | 0.9494 |
68
+ | 0.2128 | 0.4575 | 140 | 0.1408 | 0.9054 | 0.8776 | 0.8913 | 0.9616 |
69
+ | 0.1851 | 0.5229 | 160 | 0.1298 | 0.8848 | 0.8979 | 0.8913 | 0.9636 |
70
+ | 0.1812 | 0.5882 | 180 | 0.1330 | 0.8804 | 0.9242 | 0.9018 | 0.9620 |
71
+ | 0.1492 | 0.6536 | 200 | 0.1161 | 0.8742 | 0.9313 | 0.9018 | 0.9639 |
72
+ | 0.1388 | 0.7190 | 220 | 0.0979 | 0.9238 | 0.9124 | 0.9180 | 0.9707 |
73
+ | 0.1358 | 0.7843 | 240 | 0.1196 | 0.8820 | 0.9563 | 0.9177 | 0.9673 |
74
+ | 0.115 | 0.8497 | 260 | 0.0878 | 0.9164 | 0.9443 | 0.9302 | 0.9737 |
75
+ | 0.1177 | 0.9150 | 280 | 0.1110 | 0.8822 | 0.9358 | 0.9082 | 0.9660 |
76
+ | 0.1332 | 0.9804 | 300 | 0.0890 | 0.9152 | 0.9253 | 0.9202 | 0.9726 |
77
+ | 0.1079 | 1.0458 | 320 | 0.1022 | 0.8865 | 0.9379 | 0.9115 | 0.9682 |
78
+ | 0.0889 | 1.1111 | 340 | 0.1028 | 0.9014 | 0.9475 | 0.9238 | 0.9685 |
79
+ | 0.1061 | 1.1765 | 360 | 0.0958 | 0.9017 | 0.9520 | 0.9262 | 0.9709 |
80
+ | 0.1032 | 1.2418 | 380 | 0.0795 | 0.9270 | 0.9526 | 0.9396 | 0.9777 |
81
+ | 0.1034 | 1.3072 | 400 | 0.0775 | 0.9232 | 0.9520 | 0.9374 | 0.9775 |
82
+ | 0.1124 | 1.3725 | 420 | 0.0888 | 0.8985 | 0.9343 | 0.9160 | 0.9725 |
83
+ | 0.0936 | 1.4379 | 440 | 0.0870 | 0.9214 | 0.9466 | 0.9338 | 0.9756 |
84
+ | 0.0935 | 1.5033 | 460 | 0.0747 | 0.9352 | 0.9472 | 0.9412 | 0.9777 |
85
+ | 0.1005 | 1.5686 | 480 | 0.0993 | 0.9050 | 0.9579 | 0.9307 | 0.9699 |
86
+ | 0.0903 | 1.6340 | 500 | 0.0887 | 0.9067 | 0.9466 | 0.9262 | 0.9734 |
87
+ | 0.0817 | 1.6993 | 520 | 0.0775 | 0.9179 | 0.9588 | 0.9379 | 0.9780 |
88
+ | 0.0825 | 1.7647 | 540 | 0.0784 | 0.9250 | 0.9538 | 0.9392 | 0.9785 |
89
+ | 0.0921 | 1.8301 | 560 | 0.0862 | 0.9108 | 0.9635 | 0.9364 | 0.9758 |
90
+ | 0.0918 | 1.8954 | 580 | 0.0805 | 0.9194 | 0.9544 | 0.9366 | 0.9757 |
91
+ | 0.0888 | 1.9608 | 600 | 0.0819 | 0.9271 | 0.9533 | 0.9400 | 0.9770 |
92
+ | 0.0769 | 2.0261 | 620 | 0.0764 | 0.9323 | 0.9552 | 0.9436 | 0.9790 |
93
+ | 0.0683 | 2.0915 | 640 | 0.0747 | 0.9261 | 0.9579 | 0.9417 | 0.9789 |
94
+ | 0.0533 | 2.1569 | 660 | 0.0708 | 0.9309 | 0.9497 | 0.9402 | 0.9788 |
95
+ | 0.0804 | 2.2222 | 680 | 0.0732 | 0.9263 | 0.9580 | 0.9419 | 0.9783 |
96
+ | 0.0725 | 2.2876 | 700 | 0.0792 | 0.9130 | 0.9484 | 0.9304 | 0.9767 |
97
+ | 0.0562 | 2.3529 | 720 | 0.0697 | 0.9337 | 0.9586 | 0.9460 | 0.9804 |
98
+ | 0.0627 | 2.4183 | 740 | 0.0756 | 0.9349 | 0.9519 | 0.9433 | 0.9793 |
99
+ | 0.0813 | 2.4837 | 760 | 0.0740 | 0.9332 | 0.9587 | 0.9458 | 0.9805 |
100
+ | 0.0662 | 2.5490 | 780 | 0.0738 | 0.9373 | 0.9577 | 0.9474 | 0.9811 |
101
+ | 0.0728 | 2.6144 | 800 | 0.0706 | 0.9375 | 0.9576 | 0.9475 | 0.9804 |
102
+ | 0.0606 | 2.6797 | 820 | 0.0728 | 0.9415 | 0.9478 | 0.9446 | 0.9798 |
103
+ | 0.0683 | 2.7451 | 840 | 0.0717 | 0.9315 | 0.9496 | 0.9405 | 0.9797 |
104
+ | 0.0614 | 2.8105 | 860 | 0.0685 | 0.9392 | 0.9540 | 0.9466 | 0.9810 |
105
+ | 0.063 | 2.8758 | 880 | 0.0765 | 0.9287 | 0.9576 | 0.9430 | 0.9787 |
106
+ | 0.06 | 2.9412 | 900 | 0.0750 | 0.9280 | 0.9579 | 0.9427 | 0.9790 |
107
+ | 0.0605 | 3.0065 | 920 | 0.0749 | 0.9304 | 0.9616 | 0.9457 | 0.9802 |
108
+ | 0.0475 | 3.0719 | 940 | 0.0774 | 0.9249 | 0.9610 | 0.9426 | 0.9797 |
109
+ | 0.0543 | 3.1373 | 960 | 0.0845 | 0.9282 | 0.9581 | 0.9429 | 0.9779 |
110
+ | 0.0479 | 3.2026 | 980 | 0.0746 | 0.9303 | 0.9520 | 0.9410 | 0.9786 |
111
+ | 0.0575 | 3.2680 | 1000 | 0.0717 | 0.9370 | 0.9566 | 0.9467 | 0.9806 |
112
+ | 0.0498 | 3.3333 | 1020 | 0.0765 | 0.9352 | 0.9484 | 0.9418 | 0.9788 |
113
+ | 0.0539 | 3.3987 | 1040 | 0.0790 | 0.9245 | 0.9667 | 0.9451 | 0.9802 |
114
+ | 0.0583 | 3.4641 | 1060 | 0.0873 | 0.9152 | 0.9532 | 0.9338 | 0.9752 |
115
+ | 0.0557 | 3.5294 | 1080 | 0.0699 | 0.9316 | 0.9554 | 0.9433 | 0.9806 |
116
+ | 0.0507 | 3.5948 | 1100 | 0.0820 | 0.9209 | 0.9569 | 0.9385 | 0.9766 |
117
+ | 0.0472 | 3.6601 | 1120 | 0.0783 | 0.9290 | 0.9570 | 0.9428 | 0.9797 |
118
+ | 0.0685 | 3.7255 | 1140 | 0.0760 | 0.9219 | 0.9597 | 0.9404 | 0.9802 |
119
+ | 0.055 | 3.7908 | 1160 | 0.0691 | 0.9395 | 0.9601 | 0.9497 | 0.9824 |
120
+ | 0.0645 | 3.8562 | 1180 | 0.0810 | 0.9271 | 0.9569 | 0.9418 | 0.9783 |
121
+ | 0.0593 | 3.9216 | 1200 | 0.0794 | 0.9320 | 0.9563 | 0.9440 | 0.9796 |
122
+ | 0.0646 | 3.9869 | 1220 | 0.0679 | 0.9385 | 0.9610 | 0.9496 | 0.9820 |
123
+ | 0.0443 | 4.0523 | 1240 | 0.0727 | 0.9332 | 0.9575 | 0.9452 | 0.9809 |
124
+ | 0.0358 | 4.1176 | 1260 | 0.0722 | 0.9305 | 0.9552 | 0.9427 | 0.9808 |
125
+ | 0.0403 | 4.1830 | 1280 | 0.0676 | 0.9420 | 0.9629 | 0.9524 | 0.9825 |
126
+ | 0.0503 | 4.2484 | 1300 | 0.0732 | 0.9334 | 0.9599 | 0.9465 | 0.9805 |
127
+ | 0.0439 | 4.3137 | 1320 | 0.0704 | 0.9429 | 0.9524 | 0.9476 | 0.9809 |
128
+ | 0.0454 | 4.3791 | 1340 | 0.0702 | 0.9374 | 0.9572 | 0.9472 | 0.9816 |
129
+ | 0.0388 | 4.4444 | 1360 | 0.0796 | 0.9267 | 0.9558 | 0.9410 | 0.9787 |
130
+ | 0.039 | 4.5098 | 1380 | 0.0754 | 0.9305 | 0.9597 | 0.9449 | 0.9806 |
131
+ | 0.0461 | 4.5752 | 1400 | 0.0723 | 0.9350 | 0.9544 | 0.9446 | 0.9813 |
132
+ | 0.0471 | 4.6405 | 1420 | 0.0737 | 0.9318 | 0.9576 | 0.9445 | 0.9806 |
133
+ | 0.0371 | 4.7059 | 1440 | 0.0789 | 0.9290 | 0.9528 | 0.9408 | 0.9796 |
134
+ | 0.0479 | 4.7712 | 1460 | 0.0761 | 0.9287 | 0.9569 | 0.9426 | 0.9805 |
135
+ | 0.0436 | 4.8366 | 1480 | 0.0779 | 0.9313 | 0.9501 | 0.9406 | 0.9792 |
136
+ | 0.0446 | 4.9020 | 1500 | 0.0741 | 0.9387 | 0.9607 | 0.9496 | 0.9815 |
137
+ | 0.0486 | 4.9673 | 1520 | 0.0765 | 0.9299 | 0.9526 | 0.9411 | 0.9796 |
138
+ | 0.0364 | 5.0327 | 1540 | 0.0768 | 0.9351 | 0.9568 | 0.9458 | 0.9804 |
139
+ | 0.0359 | 5.0980 | 1560 | 0.0736 | 0.9399 | 0.9591 | 0.9494 | 0.9819 |
140
+ | 0.0407 | 5.1634 | 1580 | 0.0726 | 0.9413 | 0.9601 | 0.9506 | 0.9824 |
141
+ | 0.0351 | 5.2288 | 1600 | 0.0797 | 0.9249 | 0.9543 | 0.9394 | 0.9789 |
142
+ | 0.0318 | 5.2941 | 1620 | 0.0734 | 0.9393 | 0.9599 | 0.9495 | 0.9812 |
143
+ | 0.0312 | 5.3595 | 1640 | 0.0771 | 0.9366 | 0.9601 | 0.9482 | 0.9810 |
144
+ | 0.0362 | 5.4248 | 1660 | 0.0745 | 0.9387 | 0.9502 | 0.9444 | 0.9807 |
145
+ | 0.0436 | 5.4902 | 1680 | 0.0788 | 0.9299 | 0.9552 | 0.9424 | 0.9802 |
146
+ | 0.0397 | 5.5556 | 1700 | 0.0740 | 0.9389 | 0.9595 | 0.9491 | 0.9818 |
147
+ | 0.0378 | 5.6209 | 1720 | 0.0793 | 0.9312 | 0.9522 | 0.9416 | 0.9795 |
148
+ | 0.037 | 5.6863 | 1740 | 0.0753 | 0.9398 | 0.9551 | 0.9474 | 0.9809 |
149
+ | 0.0352 | 5.7516 | 1760 | 0.0807 | 0.9252 | 0.9521 | 0.9385 | 0.9791 |
150
+ | 0.0279 | 5.8170 | 1780 | 0.0769 | 0.9373 | 0.9587 | 0.9479 | 0.9810 |
151
+ | 0.037 | 5.8824 | 1800 | 0.0809 | 0.9304 | 0.9526 | 0.9414 | 0.9791 |
152
+ | 0.031 | 5.9477 | 1820 | 0.0741 | 0.9414 | 0.9609 | 0.9510 | 0.9821 |
153
+ | 0.0342 | 6.0131 | 1840 | 0.0794 | 0.9353 | 0.9558 | 0.9455 | 0.9803 |
154
+ | 0.0295 | 6.0784 | 1860 | 0.0785 | 0.9364 | 0.9531 | 0.9447 | 0.9801 |
155
+ | 0.0256 | 6.1438 | 1880 | 0.0804 | 0.9282 | 0.9512 | 0.9395 | 0.9794 |
156
+ | 0.0203 | 6.2092 | 1900 | 0.0800 | 0.9321 | 0.9507 | 0.9413 | 0.9799 |
157
+ | 0.0324 | 6.2745 | 1920 | 0.0788 | 0.9366 | 0.9568 | 0.9466 | 0.9811 |
158
+ | 0.0295 | 6.3399 | 1940 | 0.0796 | 0.9352 | 0.9570 | 0.9460 | 0.9809 |
159
+ | 0.0251 | 6.4052 | 1960 | 0.0780 | 0.9382 | 0.9558 | 0.9469 | 0.9814 |
160
+ | 0.0274 | 6.4706 | 1980 | 0.0810 | 0.9321 | 0.9524 | 0.9421 | 0.9801 |
161
+ | 0.0279 | 6.5359 | 2000 | 0.0789 | 0.9379 | 0.9583 | 0.9480 | 0.9813 |
162
+ | 0.0317 | 6.6013 | 2020 | 0.0775 | 0.9394 | 0.9569 | 0.9481 | 0.9816 |
163
+ | 0.0248 | 6.6667 | 2040 | 0.0788 | 0.9352 | 0.9542 | 0.9446 | 0.9807 |
164
+ | 0.0273 | 6.7320 | 2060 | 0.0789 | 0.9358 | 0.9557 | 0.9456 | 0.9810 |
165
+ | 0.0351 | 6.7974 | 2080 | 0.0777 | 0.9381 | 0.9573 | 0.9476 | 0.9814 |
166
+ | 0.0236 | 6.8627 | 2100 | 0.0782 | 0.9378 | 0.9569 | 0.9473 | 0.9811 |
167
+ | 0.0257 | 6.9281 | 2120 | 0.0787 | 0.9375 | 0.9573 | 0.9473 | 0.9811 |
168
+ | 0.0254 | 6.9935 | 2140 | 0.0784 | 0.9381 | 0.9575 | 0.9477 | 0.9813 |
169
+
170
+
171
+ ### Framework versions
172
+
173
+ - PEFT 0.13.2
174
+ - Transformers 4.46.3
175
+ - Pytorch 2.5.1+cu121
176
+ - Datasets 3.2.0
177
+ - Tokenizers 0.20.3