metadata
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: distilbert-base-uncased-cohl
results: []
distilbert-base-uncased-cohl
This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:
- Loss: 5.5505
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 200
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 7.4096 | 1.0 | 157 | 6.2368 |
| 6.1384 | 2.0 | 314 | 6.0235 |
| 5.9964 | 3.0 | 471 | 5.9395 |
| 5.9145 | 4.0 | 628 | 5.8933 |
| 5.8601 | 5.0 | 785 | 5.8380 |
| 5.8461 | 6.0 | 942 | 5.7921 |
| 5.82 | 7.0 | 1099 | 5.7787 |
| 5.8076 | 8.0 | 1256 | 5.7794 |
| 5.7927 | 9.0 | 1413 | 5.7946 |
| 5.782 | 10.0 | 1570 | 5.7553 |
| 5.7691 | 11.0 | 1727 | 5.7753 |
| 5.7671 | 12.0 | 1884 | 5.7607 |
| 5.7594 | 13.0 | 2041 | 5.7564 |
| 5.7443 | 14.0 | 2198 | 5.7553 |
| 5.7354 | 15.0 | 2355 | 5.7421 |
| 5.7428 | 16.0 | 2512 | 5.7304 |
| 5.7319 | 17.0 | 2669 | 5.7053 |
| 5.7187 | 18.0 | 2826 | 5.7095 |
| 5.7273 | 19.0 | 2983 | 5.7034 |
| 5.7121 | 20.0 | 3140 | 5.6822 |
| 5.7139 | 21.0 | 3297 | 5.7028 |
| 5.7072 | 22.0 | 3454 | 5.7020 |
| 5.695 | 23.0 | 3611 | 5.7085 |
| 5.6921 | 24.0 | 3768 | 5.6935 |
| 5.6964 | 25.0 | 3925 | 5.7071 |
| 5.6771 | 26.0 | 4082 | 5.7016 |
| 5.6911 | 27.0 | 4239 | 5.6765 |
| 5.6874 | 28.0 | 4396 | 5.6937 |
| 5.6788 | 29.0 | 4553 | 5.6744 |
| 5.6709 | 30.0 | 4710 | 5.6593 |
| 5.6743 | 31.0 | 4867 | 5.6719 |
| 5.6623 | 32.0 | 5024 | 5.6422 |
| 5.662 | 33.0 | 5181 | 5.6660 |
| 5.6577 | 34.0 | 5338 | 5.6790 |
| 5.6603 | 35.0 | 5495 | 5.6556 |
| 5.6607 | 36.0 | 5652 | 5.6476 |
| 5.6538 | 37.0 | 5809 | 5.6643 |
| 5.6481 | 38.0 | 5966 | 5.6489 |
| 5.6512 | 39.0 | 6123 | 5.6108 |
| 5.642 | 40.0 | 6280 | 5.6647 |
| 5.6475 | 41.0 | 6437 | 5.6633 |
| 5.6419 | 42.0 | 6594 | 5.6256 |
| 5.6364 | 43.0 | 6751 | 5.6524 |
| 5.6391 | 44.0 | 6908 | 5.6424 |
| 5.6307 | 45.0 | 7065 | 5.6384 |
| 5.6249 | 46.0 | 7222 | 5.6451 |
| 5.6242 | 47.0 | 7379 | 5.6413 |
| 5.6259 | 48.0 | 7536 | 5.6230 |
| 5.6223 | 49.0 | 7693 | 5.6285 |
| 5.6245 | 50.0 | 7850 | 5.6107 |
| 5.621 | 51.0 | 8007 | 5.6253 |
| 5.6203 | 52.0 | 8164 | 5.6457 |
| 5.6131 | 53.0 | 8321 | 5.6211 |
| 5.6026 | 54.0 | 8478 | 5.6360 |
| 5.6115 | 55.0 | 8635 | 5.6276 |
| 5.6079 | 56.0 | 8792 | 5.6274 |
| 5.6106 | 57.0 | 8949 | 5.6289 |
| 5.6053 | 58.0 | 9106 | 5.6438 |
| 5.6113 | 59.0 | 9263 | 5.6258 |
| 5.5983 | 60.0 | 9420 | 5.6453 |
| 5.6 | 61.0 | 9577 | 5.6351 |
| 5.6007 | 62.0 | 9734 | 5.6327 |
| 5.5989 | 63.0 | 9891 | 5.6102 |
| 5.5974 | 64.0 | 10048 | 5.6280 |
| 5.5987 | 65.0 | 10205 | 5.6299 |
| 5.5903 | 66.0 | 10362 | 5.6106 |
| 5.5915 | 67.0 | 10519 | 5.6149 |
| 5.5928 | 68.0 | 10676 | 5.6048 |
| 5.5876 | 69.0 | 10833 | 5.6279 |
| 5.5886 | 70.0 | 10990 | 5.6073 |
| 5.5859 | 71.0 | 11147 | 5.5987 |
| 5.5881 | 72.0 | 11304 | 5.6208 |
| 5.5805 | 73.0 | 11461 | 5.5869 |
| 5.5808 | 74.0 | 11618 | 5.6169 |
| 5.5813 | 75.0 | 11775 | 5.6019 |
| 5.5881 | 76.0 | 11932 | 5.6213 |
| 5.5823 | 77.0 | 12089 | 5.5931 |
| 5.5735 | 78.0 | 12246 | 5.5948 |
| 5.5788 | 79.0 | 12403 | 5.5878 |
| 5.5735 | 80.0 | 12560 | 5.5784 |
| 5.5701 | 81.0 | 12717 | 5.6084 |
| 5.5757 | 82.0 | 12874 | 5.5957 |
| 5.5697 | 83.0 | 13031 | 5.5931 |
| 5.573 | 84.0 | 13188 | 5.5862 |
| 5.5652 | 85.0 | 13345 | 5.6049 |
| 5.5635 | 86.0 | 13502 | 5.5959 |
| 5.5634 | 87.0 | 13659 | 5.5865 |
| 5.5644 | 88.0 | 13816 | 5.6000 |
| 5.5662 | 89.0 | 13973 | 5.5971 |
| 5.5563 | 90.0 | 14130 | 5.5711 |
| 5.5612 | 91.0 | 14287 | 5.6007 |
| 5.5626 | 92.0 | 14444 | 5.5824 |
| 5.5543 | 93.0 | 14601 | 5.5966 |
| 5.5627 | 94.0 | 14758 | 5.5828 |
| 5.5633 | 95.0 | 14915 | 5.6066 |
| 5.5526 | 96.0 | 15072 | 5.5979 |
| 5.5529 | 97.0 | 15229 | 5.5756 |
| 5.5527 | 98.0 | 15386 | 5.5633 |
| 5.5568 | 99.0 | 15543 | 5.5775 |
| 5.5419 | 100.0 | 15700 | 5.5899 |
| 5.5436 | 101.0 | 15857 | 5.5657 |
| 5.5509 | 102.0 | 16014 | 5.5824 |
| 5.5468 | 103.0 | 16171 | 5.5936 |
| 5.5447 | 104.0 | 16328 | 5.5666 |
| 5.5469 | 105.0 | 16485 | 5.5747 |
| 5.5436 | 106.0 | 16642 | 5.5658 |
| 5.537 | 107.0 | 16799 | 5.5873 |
| 5.5356 | 108.0 | 16956 | 5.5981 |
| 5.5355 | 109.0 | 17113 | 5.5884 |
| 5.539 | 110.0 | 17270 | 5.5713 |
| 5.5413 | 111.0 | 17427 | 5.5951 |
| 5.5353 | 112.0 | 17584 | 5.5817 |
| 5.5275 | 113.0 | 17741 | 5.5981 |
| 5.5422 | 114.0 | 17898 | 5.5744 |
| 5.5298 | 115.0 | 18055 | 5.5637 |
| 5.5335 | 116.0 | 18212 | 5.5918 |
| 5.5305 | 117.0 | 18369 | 5.5717 |
| 5.5257 | 118.0 | 18526 | 5.5681 |
| 5.5313 | 119.0 | 18683 | 5.5984 |
| 5.5286 | 120.0 | 18840 | 5.5799 |
| 5.5217 | 121.0 | 18997 | 5.5746 |
| 5.5309 | 122.0 | 19154 | 5.5429 |
| 5.5288 | 123.0 | 19311 | 5.5787 |
| 5.5258 | 124.0 | 19468 | 5.5942 |
| 5.5185 | 125.0 | 19625 | 5.5922 |
| 5.5232 | 126.0 | 19782 | 5.5587 |
| 5.5227 | 127.0 | 19939 | 5.5575 |
| 5.5356 | 128.0 | 20096 | 5.5800 |
| 5.5226 | 129.0 | 20253 | 5.5780 |
| 5.5243 | 130.0 | 20410 | 5.5717 |
| 5.5154 | 131.0 | 20567 | 5.5644 |
| 5.5216 | 132.0 | 20724 | 5.5741 |
| 5.5212 | 133.0 | 20881 | 5.5778 |
| 5.5154 | 134.0 | 21038 | 5.5588 |
| 5.5124 | 135.0 | 21195 | 5.5647 |
| 5.5164 | 136.0 | 21352 | 5.5449 |
| 5.5176 | 137.0 | 21509 | 5.5625 |
| 5.5078 | 138.0 | 21666 | 5.5803 |
| 5.5137 | 139.0 | 21823 | 5.5805 |
| 5.5154 | 140.0 | 21980 | 5.5494 |
| 5.5188 | 141.0 | 22137 | 5.5791 |
| 5.5032 | 142.0 | 22294 | 5.5724 |
| 5.509 | 143.0 | 22451 | 5.5921 |
| 5.5112 | 144.0 | 22608 | 5.5688 |
| 5.5041 | 145.0 | 22765 | 5.5619 |
| 5.5103 | 146.0 | 22922 | 5.5735 |
| 5.5112 | 147.0 | 23079 | 5.5763 |
| 5.5085 | 148.0 | 23236 | 5.5748 |
| 5.506 | 149.0 | 23393 | 5.5738 |
| 5.5118 | 150.0 | 23550 | 5.5718 |
| 5.5014 | 151.0 | 23707 | 5.5619 |
| 5.5087 | 152.0 | 23864 | 5.5810 |
| 5.51 | 153.0 | 24021 | 5.5804 |
| 5.5028 | 154.0 | 24178 | 5.5870 |
| 5.5157 | 155.0 | 24335 | 5.5536 |
| 5.5043 | 156.0 | 24492 | 5.5856 |
| 5.5083 | 157.0 | 24649 | 5.5663 |
| 5.5014 | 158.0 | 24806 | 5.5883 |
| 5.4994 | 159.0 | 24963 | 5.5754 |
| 5.5025 | 160.0 | 25120 | 5.5567 |
| 5.4998 | 161.0 | 25277 | 5.5729 |
| 5.5009 | 162.0 | 25434 | 5.5422 |
| 5.5063 | 163.0 | 25591 | 5.5731 |
| 5.5093 | 164.0 | 25748 | 5.5734 |
| 5.5011 | 165.0 | 25905 | 5.5617 |
| 5.5011 | 166.0 | 26062 | 5.5586 |
| 5.5017 | 167.0 | 26219 | 5.5483 |
| 5.5001 | 168.0 | 26376 | 5.5617 |
| 5.4964 | 169.0 | 26533 | 5.5477 |
| 5.5014 | 170.0 | 26690 | 5.5646 |
| 5.4981 | 171.0 | 26847 | 5.5723 |
| 5.4902 | 172.0 | 27004 | 5.5530 |
| 5.4957 | 173.0 | 27161 | 5.5614 |
| 5.4988 | 174.0 | 27318 | 5.5699 |
| 5.5005 | 175.0 | 27475 | 5.5637 |
| 5.5005 | 176.0 | 27632 | 5.5769 |
| 5.4973 | 177.0 | 27789 | 5.5624 |
| 5.4927 | 178.0 | 27946 | 5.5736 |
| 5.4962 | 179.0 | 28103 | 5.5639 |
| 5.4908 | 180.0 | 28260 | 5.5541 |
| 5.4909 | 181.0 | 28417 | 5.5598 |
| 5.4885 | 182.0 | 28574 | 5.5642 |
| 5.4902 | 183.0 | 28731 | 5.5590 |
| 5.4949 | 184.0 | 28888 | 5.5707 |
| 5.4935 | 185.0 | 29045 | 5.5597 |
| 5.4914 | 186.0 | 29202 | 5.5823 |
| 5.4914 | 187.0 | 29359 | 5.5597 |
| 5.4874 | 188.0 | 29516 | 5.5595 |
| 5.4934 | 189.0 | 29673 | 5.5685 |
| 5.4956 | 190.0 | 29830 | 5.5578 |
| 5.4902 | 191.0 | 29987 | 5.5762 |
| 5.4881 | 192.0 | 30144 | 5.5697 |
| 5.4934 | 193.0 | 30301 | 5.5631 |
| 5.4974 | 194.0 | 30458 | 5.5730 |
| 5.4939 | 195.0 | 30615 | 5.5614 |
| 5.4952 | 196.0 | 30772 | 5.5492 |
| 5.4892 | 197.0 | 30929 | 5.5613 |
| 5.49 | 198.0 | 31086 | 5.5737 |
| 5.4914 | 199.0 | 31243 | 5.5806 |
| 5.4954 | 200.0 | 31400 | 5.5505 |
Framework versions
- Transformers 4.30.2
- Pytorch 2.0.1+cu118
- Datasets 2.13.0
- Tokenizers 0.13.3