Hugging Face's logo

sickcell
/
bert-base-uncased-finetuned-github_cybersecurity_READMEs

Fill-Mask
Transformers
TensorBoard
Safetensors
bert
Generated from Trainer
Model card Files
xet
Metrics Community
sickcell69 commited on
Commit
3266f2f
·
verified ·
1 Parent(s): 2706cdc

End of training

Browse files
Files changed (4) hide show
  1. README.md +110 -15
  2. model.safetensors +1 -1
  3. runs/Apr13_19-43-14_DESKTOP-7EBBP1S/events.out.tfevents.1713008595.DESKTOP-7EBBP1S.227770.2 +2 -2
  4. runs/Apr13_19-43-14_DESKTOP-7EBBP1S/events.out.tfevents.1713009887.DESKTOP-7EBBP1S.227770.3 +3 -0
README.md CHANGED
@@ -3,6 +3,8 @@ license: apache-2.0
3
  base_model: bert-base-uncased
4
  tags:
5
  - generated_from_trainer
 
 
6
  model-index:
7
  - name: bert-base-uncased-finetuned-github_cybersecurity_READMEs
8
  results: []
@@ -15,7 +17,8 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 4.7290
 
19
 
20
  ## Model description
21
 
@@ -34,26 +37,118 @@ More information needed
34
  ### Training hyperparameters
35
 
36
  The following hyperparameters were used during training:
37
- - learning_rate: 2e-05
38
- - train_batch_size: 16
39
- - eval_batch_size: 16
40
  - seed: 42
41
- - gradient_accumulation_steps: 2
42
- - total_train_batch_size: 32
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
  - lr_scheduler_type: linear
45
- - lr_scheduler_warmup_steps: 500
46
- - num_epochs: 5
47
 
48
  ### Training results
49
 
50
- | Training Loss | Epoch | Step | Validation Loss |
51
- |:-------------:|:-----:|:----:|:---------------:|
52
- | No log | 1.0 | 58 | 8.0800 |
53
- | No log | 2.0 | 116 | 6.5805 |
54
- | No log | 3.0 | 174 | 5.9328 |
55
- | No log | 4.0 | 232 | 5.1172 |
56
- | No log | 5.0 | 290 | 4.6694 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
 
58
 
59
  ### Framework versions
 
3
  base_model: bert-base-uncased
4
  tags:
5
  - generated_from_trainer
6
+ metrics:
7
+ - accuracy
8
  model-index:
9
  - name: bert-base-uncased-finetuned-github_cybersecurity_READMEs
10
  results: []
 
17
 
18
  This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 2.2291
21
+ - Accuracy: 0.6479
22
 
23
  ## Model description
24
 
 
37
  ### Training hyperparameters
38
 
39
  The following hyperparameters were used during training:
40
+ - learning_rate: 3e-05
41
+ - train_batch_size: 32
42
+ - eval_batch_size: 32
43
  - seed: 42
44
+ - gradient_accumulation_steps: 4
45
+ - total_train_batch_size: 128
46
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
47
  - lr_scheduler_type: linear
48
+ - lr_scheduler_warmup_steps: 1000
49
+ - num_epochs: 100
50
 
51
  ### Training results
52
 
53
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
54
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|
55
+ | No log | 0.97 | 14 | 4.1856 | 0.4305 |
56
+ | No log | 2.0 | 29 | 4.3178 | 0.4090 |
57
+ | No log | 2.97 | 43 | 4.0734 | 0.4342 |
58
+ | No log | 4.0 | 58 | 4.0470 | 0.4332 |
59
+ | No log | 4.97 | 72 | 4.0668 | 0.4270 |
60
+ | No log | 6.0 | 87 | 3.9068 | 0.4390 |
61
+ | No log | 6.97 | 101 | 3.8466 | 0.4468 |
62
+ | No log | 8.0 | 116 | 3.8330 | 0.4535 |
63
+ | No log | 8.97 | 130 | 3.7238 | 0.4516 |
64
+ | No log | 10.0 | 145 | 3.8113 | 0.4446 |
65
+ | No log | 10.97 | 159 | 3.6681 | 0.4607 |
66
+ | No log | 12.0 | 174 | 3.5627 | 0.4679 |
67
+ | No log | 12.97 | 188 | 3.4540 | 0.4794 |
68
+ | No log | 14.0 | 203 | 3.5997 | 0.4707 |
69
+ | No log | 14.97 | 217 | 3.4362 | 0.4860 |
70
+ | No log | 16.0 | 232 | 3.5471 | 0.4740 |
71
+ | No log | 16.97 | 246 | 3.4968 | 0.4803 |
72
+ | No log | 18.0 | 261 | 3.2938 | 0.4985 |
73
+ | No log | 18.97 | 275 | 3.4207 | 0.4765 |
74
+ | No log | 20.0 | 290 | 3.3869 | 0.4970 |
75
+ | No log | 20.97 | 304 | 3.3062 | 0.5012 |
76
+ | No log | 22.0 | 319 | 3.3184 | 0.4917 |
77
+ | No log | 22.97 | 333 | 3.2132 | 0.5136 |
78
+ | No log | 24.0 | 348 | 3.2027 | 0.5074 |
79
+ | No log | 24.97 | 362 | 3.3251 | 0.4923 |
80
+ | No log | 26.0 | 377 | 3.1569 | 0.5108 |
81
+ | No log | 26.97 | 391 | 3.0947 | 0.5194 |
82
+ | No log | 28.0 | 406 | 3.0470 | 0.5206 |
83
+ | No log | 28.97 | 420 | 3.0662 | 0.5182 |
84
+ | No log | 30.0 | 435 | 3.0845 | 0.5191 |
85
+ | No log | 30.97 | 449 | 3.0681 | 0.5219 |
86
+ | No log | 32.0 | 464 | 2.9902 | 0.5263 |
87
+ | No log | 32.97 | 478 | 2.8970 | 0.5448 |
88
+ | No log | 34.0 | 493 | 2.9269 | 0.5341 |
89
+ | 3.629 | 34.97 | 507 | 2.8605 | 0.5519 |
90
+ | 3.629 | 36.0 | 522 | 2.8657 | 0.5431 |
91
+ | 3.629 | 36.97 | 536 | 2.9391 | 0.5407 |
92
+ | 3.629 | 38.0 | 551 | 2.8960 | 0.5437 |
93
+ | 3.629 | 38.97 | 565 | 2.8819 | 0.5466 |
94
+ | 3.629 | 40.0 | 580 | 2.7555 | 0.5633 |
95
+ | 3.629 | 40.97 | 594 | 2.7425 | 0.5555 |
96
+ | 3.629 | 42.0 | 609 | 2.7960 | 0.5615 |
97
+ | 3.629 | 42.97 | 623 | 2.7382 | 0.5630 |
98
+ | 3.629 | 44.0 | 638 | 2.7967 | 0.5580 |
99
+ | 3.629 | 44.97 | 652 | 2.6611 | 0.5781 |
100
+ | 3.629 | 46.0 | 667 | 2.6877 | 0.5722 |
101
+ | 3.629 | 46.97 | 681 | 2.7917 | 0.5609 |
102
+ | 3.629 | 48.0 | 696 | 2.7029 | 0.5696 |
103
+ | 3.629 | 48.97 | 710 | 2.7408 | 0.5618 |
104
+ | 3.629 | 50.0 | 725 | 2.6450 | 0.5772 |
105
+ | 3.629 | 50.97 | 739 | 2.5569 | 0.5883 |
106
+ | 3.629 | 52.0 | 754 | 2.6646 | 0.5795 |
107
+ | 3.629 | 52.97 | 768 | 2.6803 | 0.5729 |
108
+ | 3.629 | 54.0 | 783 | 2.6233 | 0.5847 |
109
+ | 3.629 | 54.97 | 797 | 2.6027 | 0.5842 |
110
+ | 3.629 | 56.0 | 812 | 2.4090 | 0.6034 |
111
+ | 3.629 | 56.97 | 826 | 2.4978 | 0.6011 |
112
+ | 3.629 | 58.0 | 841 | 2.5106 | 0.5944 |
113
+ | 3.629 | 58.97 | 855 | 2.5039 | 0.5912 |
114
+ | 3.629 | 60.0 | 870 | 2.5792 | 0.5824 |
115
+ | 3.629 | 60.97 | 884 | 2.4764 | 0.6065 |
116
+ | 3.629 | 62.0 | 899 | 2.5348 | 0.6036 |
117
+ | 3.629 | 62.97 | 913 | 2.5338 | 0.6022 |
118
+ | 3.629 | 64.0 | 928 | 2.4646 | 0.6130 |
119
+ | 3.629 | 64.97 | 942 | 2.4532 | 0.6066 |
120
+ | 3.629 | 66.0 | 957 | 2.4526 | 0.6073 |
121
+ | 3.629 | 66.97 | 971 | 2.5369 | 0.5992 |
122
+ | 3.629 | 68.0 | 986 | 2.4170 | 0.6181 |
123
+ | 2.5556 | 68.97 | 1000 | 2.4493 | 0.6078 |
124
+ | 2.5556 | 70.0 | 1015 | 2.3939 | 0.6159 |
125
+ | 2.5556 | 70.97 | 1029 | 2.4793 | 0.6049 |
126
+ | 2.5556 | 72.0 | 1044 | 2.3225 | 0.6286 |
127
+ | 2.5556 | 72.97 | 1058 | 2.3551 | 0.6212 |
128
+ | 2.5556 | 74.0 | 1073 | 2.4702 | 0.6075 |
129
+ | 2.5556 | 74.97 | 1087 | 2.3489 | 0.6311 |
130
+ | 2.5556 | 76.0 | 1102 | 2.3455 | 0.6198 |
131
+ | 2.5556 | 76.97 | 1116 | 2.4500 | 0.6145 |
132
+ | 2.5556 | 78.0 | 1131 | 2.3223 | 0.6332 |
133
+ | 2.5556 | 78.97 | 1145 | 2.4375 | 0.6065 |
134
+ | 2.5556 | 80.0 | 1160 | 2.2743 | 0.6291 |
135
+ | 2.5556 | 80.97 | 1174 | 2.3255 | 0.6295 |
136
+ | 2.5556 | 82.0 | 1189 | 2.3785 | 0.6237 |
137
+ | 2.5556 | 82.97 | 1203 | 2.2722 | 0.6344 |
138
+ | 2.5556 | 84.0 | 1218 | 2.2392 | 0.6407 |
139
+ | 2.5556 | 84.97 | 1232 | 2.2322 | 0.6361 |
140
+ | 2.5556 | 86.0 | 1247 | 2.2206 | 0.6496 |
141
+ | 2.5556 | 86.97 | 1261 | 2.2419 | 0.6345 |
142
+ | 2.5556 | 88.0 | 1276 | 2.1919 | 0.6492 |
143
+ | 2.5556 | 88.97 | 1290 | 2.2616 | 0.6433 |
144
+ | 2.5556 | 90.0 | 1305 | 2.2227 | 0.6417 |
145
+ | 2.5556 | 90.97 | 1319 | 2.2847 | 0.6447 |
146
+ | 2.5556 | 92.0 | 1334 | 2.2916 | 0.6339 |
147
+ | 2.5556 | 92.97 | 1348 | 2.2684 | 0.6410 |
148
+ | 2.5556 | 94.0 | 1363 | 2.2432 | 0.6440 |
149
+ | 2.5556 | 94.97 | 1377 | 2.2510 | 0.6462 |
150
+ | 2.5556 | 96.0 | 1392 | 2.2970 | 0.6363 |
151
+ | 2.5556 | 96.55 | 1400 | 2.2197 | 0.6423 |
152
 
153
 
154
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8954ca53f641201f151832d0500a8967f0f0d66fe3ad01545cba4785da06baad
3
  size 438080896
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:65d4c3bb10b745664e18f5777825351a2e24841337792fc255516367d2fa078d
3
  size 438080896
runs/Apr13_19-43-14_DESKTOP-7EBBP1S/events.out.tfevents.1713008595.DESKTOP-7EBBP1S.227770.2 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:73b5e37fba8961ed00d7fb76fe88369f415923f46f53a545075c0ac092a4126a
3
- size 15862
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c14a796135cd503a9b1b1a481921eb324d0c533eda9d9089b5c9c9e8b7eccf54
3
+ size 36776
runs/Apr13_19-43-14_DESKTOP-7EBBP1S/events.out.tfevents.1713009887.DESKTOP-7EBBP1S.227770.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:042034758ab5fe9f46411cc0faf16c0417180d1eb9dafb5ccbce2d6701663d2d
3
+ size 411