--- license: mit base_model: roberta-base tags: - generated_from_trainer metrics: - precision - recall - f1 - accuracy model-index: - name: roberta-base-finetuned-ner-cadec-no-iob results: [] --- # roberta-base-finetuned-ner-cadec-no-iob This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.4142 - Precision: 0.6452 - Recall: 0.6860 - F1: 0.6650 - Accuracy: 0.9380 - Adr Precision: 0.5911 - Adr Recall: 0.6557 - Adr F1: 0.6217 - Disease Precision: 0.4138 - Disease Recall: 0.375 - Disease F1: 0.3934 - Drug Precision: 0.8962 - Drug Recall: 0.9111 - Drug F1: 0.9036 - Finding Precision: 0.375 - Finding Recall: 0.375 - Finding F1: 0.375 - Symptom Precision: 0.5833 - Symptom Recall: 0.4828 - Symptom F1: 0.5283 - Macro Avg F1: 0.5644 - Weighted Avg F1: 0.6650 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 35 ### Training results | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy | Adr Precision | Adr Recall | Adr F1 | Disease Precision | Disease Recall | Disease F1 | Drug Precision | Drug Recall | Drug F1 | Finding Precision | Finding Recall | Finding F1 | Symptom Precision | Symptom Recall | Symptom F1 | Macro Avg F1 | Weighted Avg F1 | |:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|:-------------:|:----------:|:------:|:-----------------:|:--------------:|:----------:|:--------------:|:-----------:|:-------:|:-----------------:|:--------------:|:----------:|:-----------------:|:--------------:|:----------:|:------------:|:---------------:| | No log | 1.0 | 125 | 0.2142 | 0.5325 | 0.6055 | 0.5667 | 0.9194 | 0.4548 | 0.5918 | 0.5143 | 0.4186 | 0.5625 | 0.48 | 0.8398 | 0.8444 | 0.8421 | 0.2857 | 0.0625 | 0.1026 | 0.0 | 0.0 | 0.0 | 0.3878 | 0.5537 | | No log | 2.0 | 250 | 0.1798 | 0.6083 | 0.6557 | 0.6311 | 0.9339 | 0.5276 | 0.6309 | 0.5746 | 0.6842 | 0.4062 | 0.5098 | 0.8950 | 0.9 | 0.8975 | 0.32 | 0.25 | 0.2807 | 0.6667 | 0.2759 | 0.3902 | 0.5306 | 0.6291 | | No log | 3.0 | 375 | 0.1910 | 0.5748 | 0.6029 | 0.5885 | 0.9282 | 0.5191 | 0.5320 | 0.5255 | 0.6 | 0.375 | 0.4615 | 0.8820 | 0.8722 | 0.8771 | 0.2927 | 0.375 | 0.3288 | 0.3051 | 0.6207 | 0.4091 | 0.5204 | 0.5935 | | 0.1902 | 4.0 | 500 | 0.2013 | 0.5995 | 0.6398 | 0.6190 | 0.9311 | 0.5460 | 0.6 | 0.5717 | 0.25 | 0.0938 | 0.1364 | 0.8840 | 0.8889 | 0.8864 | 0.2632 | 0.4688 | 0.3371 | 0.6154 | 0.5517 | 0.5818 | 0.5027 | 0.6185 | | 0.1902 | 5.0 | 625 | 0.2113 | 0.6161 | 0.6649 | 0.6396 | 0.9335 | 0.5515 | 0.6289 | 0.5877 | 0.5556 | 0.4688 | 0.5085 | 0.8852 | 0.9 | 0.8926 | 0.2857 | 0.25 | 0.2667 | 0.5185 | 0.4828 | 0.5 | 0.5511 | 0.6398 | | 0.1902 | 6.0 | 750 | 0.1955 | 0.6223 | 0.6544 | 0.6379 | 0.9341 | 0.5541 | 0.6021 | 0.5771 | 0.5833 | 0.4375 | 0.5 | 0.8956 | 0.9056 | 0.9006 | 0.2857 | 0.375 | 0.3243 | 0.6818 | 0.5172 | 0.5882 | 0.5780 | 0.6404 | | 0.1902 | 7.0 | 875 | 0.2226 | 0.6252 | 0.6491 | 0.6369 | 0.9343 | 0.5671 | 0.6186 | 0.5917 | 0.5556 | 0.1562 | 0.2439 | 0.8983 | 0.8833 | 0.8908 | 0.3061 | 0.4688 | 0.3704 | 0.5652 | 0.4483 | 0.5000 | 0.5193 | 0.6352 | | 0.0648 | 8.0 | 1000 | 0.2345 | 0.6229 | 0.6755 | 0.6481 | 0.9363 | 0.5773 | 0.6392 | 0.6067 | 0.4138 | 0.375 | 0.3934 | 0.875 | 0.8944 | 0.8846 | 0.2973 | 0.3438 | 0.3188 | 0.5143 | 0.6207 | 0.5625 | 0.5532 | 0.6498 | | 0.0648 | 9.0 | 1125 | 0.2316 | 0.6322 | 0.6689 | 0.65 | 0.9368 | 0.5851 | 0.6309 | 0.6071 | 0.5 | 0.3125 | 0.3846 | 0.8811 | 0.9056 | 0.8932 | 0.2766 | 0.4062 | 0.3291 | 0.5556 | 0.5172 | 0.5357 | 0.5499 | 0.6512 | | 0.0648 | 10.0 | 1250 | 0.2944 | 0.6204 | 0.6491 | 0.6344 | 0.9320 | 0.5551 | 0.6021 | 0.5776 | 0.5789 | 0.3438 | 0.4314 | 0.8913 | 0.9111 | 0.9011 | 0.2619 | 0.3438 | 0.2973 | 0.6364 | 0.4828 | 0.5490 | 0.5513 | 0.6353 | | 0.0648 | 11.0 | 1375 | 0.2660 | 0.6280 | 0.6794 | 0.6527 | 0.9353 | 0.5786 | 0.6454 | 0.6101 | 0.3824 | 0.4062 | 0.3939 | 0.8956 | 0.9056 | 0.9006 | 0.2812 | 0.2812 | 0.2812 | 0.5484 | 0.5862 | 0.5667 | 0.5505 | 0.6544 | | 0.0284 | 12.0 | 1500 | 0.2819 | 0.6366 | 0.6702 | 0.6530 | 0.9355 | 0.5827 | 0.6392 | 0.6096 | 0.5 | 0.1875 | 0.2727 | 0.8956 | 0.9056 | 0.9006 | 0.3478 | 0.5 | 0.4103 | 0.5 | 0.4483 | 0.4727 | 0.5332 | 0.6508 | | 0.0284 | 13.0 | 1625 | 0.3000 | 0.6326 | 0.6702 | 0.6509 | 0.9363 | 0.5736 | 0.6351 | 0.6027 | 0.4444 | 0.375 | 0.4068 | 0.8950 | 0.9 | 0.8975 | 0.3421 | 0.4062 | 0.3714 | 0.65 | 0.4483 | 0.5306 | 0.5618 | 0.6519 | | 0.0284 | 14.0 | 1750 | 0.2996 | 0.6228 | 0.6491 | 0.6357 | 0.9363 | 0.5645 | 0.6041 | 0.5837 | 0.4783 | 0.3438 | 0.4 | 0.8743 | 0.8889 | 0.8815 | 0.2973 | 0.3438 | 0.3188 | 0.6071 | 0.5862 | 0.5965 | 0.5561 | 0.6360 | | 0.0284 | 15.0 | 1875 | 0.3246 | 0.6311 | 0.6636 | 0.6469 | 0.9352 | 0.5788 | 0.6206 | 0.5990 | 0.45 | 0.2812 | 0.3462 | 0.9056 | 0.9056 | 0.9056 | 0.2683 | 0.3438 | 0.3014 | 0.5278 | 0.6552 | 0.5846 | 0.5473 | 0.6480 | | 0.0136 | 16.0 | 2000 | 0.3305 | 0.6461 | 0.6623 | 0.6541 | 0.9377 | 0.5869 | 0.6268 | 0.6062 | 0.4545 | 0.3125 | 0.3704 | 0.9011 | 0.9111 | 0.9061 | 0.3448 | 0.3125 | 0.3279 | 0.5385 | 0.4828 | 0.5091 | 0.5439 | 0.6520 | | 0.0136 | 17.0 | 2125 | 0.3181 | 0.6291 | 0.6781 | 0.6527 | 0.9375 | 0.5780 | 0.6495 | 0.6117 | 0.4231 | 0.3438 | 0.3793 | 0.9066 | 0.9167 | 0.9116 | 0.2857 | 0.3125 | 0.2985 | 0.4483 | 0.4483 | 0.4483 | 0.5299 | 0.6536 | | 0.0136 | 18.0 | 2250 | 0.3414 | 0.6298 | 0.6755 | 0.6518 | 0.9362 | 0.5765 | 0.6371 | 0.6053 | 0.375 | 0.375 | 0.375 | 0.8962 | 0.9111 | 0.9036 | 0.3235 | 0.3438 | 0.3333 | 0.5714 | 0.5517 | 0.5614 | 0.5557 | 0.6532 | | 0.0136 | 19.0 | 2375 | 0.3457 | 0.6302 | 0.6768 | 0.6527 | 0.9372 | 0.5877 | 0.6495 | 0.6170 | 0.3636 | 0.25 | 0.2963 | 0.8907 | 0.9056 | 0.8981 | 0.26 | 0.4062 | 0.3171 | 0.6087 | 0.4828 | 0.5385 | 0.5334 | 0.6546 | | 0.0078 | 20.0 | 2500 | 0.3700 | 0.6367 | 0.6636 | 0.6499 | 0.9367 | 0.5805 | 0.6247 | 0.6018 | 0.3714 | 0.4062 | 0.3881 | 0.9016 | 0.9167 | 0.9091 | 0.3077 | 0.25 | 0.2759 | 0.5833 | 0.4828 | 0.5283 | 0.5406 | 0.6492 | | 0.0078 | 21.0 | 2625 | 0.3772 | 0.6276 | 0.6715 | 0.6488 | 0.9325 | 0.5766 | 0.6289 | 0.6016 | 0.44 | 0.3438 | 0.3860 | 0.8919 | 0.9167 | 0.9041 | 0.2927 | 0.375 | 0.3288 | 0.5161 | 0.5517 | 0.5333 | 0.5508 | 0.6502 | | 0.0078 | 22.0 | 2750 | 0.3622 | 0.6389 | 0.6768 | 0.6573 | 0.9345 | 0.5855 | 0.6495 | 0.6158 | 0.4333 | 0.4062 | 0.4194 | 0.8840 | 0.8889 | 0.8864 | 0.3333 | 0.3125 | 0.3226 | 0.625 | 0.5172 | 0.5660 | 0.5620 | 0.6575 | | 0.0078 | 23.0 | 2875 | 0.3811 | 0.6304 | 0.6728 | 0.6509 | 0.9352 | 0.5765 | 0.6371 | 0.6053 | 0.4 | 0.375 | 0.3871 | 0.8804 | 0.9 | 0.8901 | 0.3438 | 0.3438 | 0.3438 | 0.5926 | 0.5517 | 0.5714 | 0.5595 | 0.6514 | | 0.005 | 24.0 | 3000 | 0.3824 | 0.6322 | 0.6689 | 0.65 | 0.9353 | 0.5757 | 0.6351 | 0.6039 | 0.4286 | 0.375 | 0.4000 | 0.8901 | 0.9 | 0.8950 | 0.3226 | 0.3125 | 0.3175 | 0.5769 | 0.5172 | 0.5455 | 0.5524 | 0.6501 | | 0.005 | 25.0 | 3125 | 0.3821 | 0.6297 | 0.6821 | 0.6548 | 0.9375 | 0.5850 | 0.6598 | 0.6202 | 0.4 | 0.375 | 0.3871 | 0.8852 | 0.9 | 0.8926 | 0.25 | 0.2812 | 0.2647 | 0.56 | 0.4828 | 0.5185 | 0.5366 | 0.6561 | | 0.005 | 26.0 | 3250 | 0.4058 | 0.6292 | 0.6715 | 0.6496 | 0.9355 | 0.5821 | 0.6433 | 0.6112 | 0.3939 | 0.4062 | 0.4 | 0.875 | 0.8944 | 0.8846 | 0.2857 | 0.25 | 0.2667 | 0.5357 | 0.5172 | 0.5263 | 0.5378 | 0.6494 | | 0.005 | 27.0 | 3375 | 0.3980 | 0.6262 | 0.6807 | 0.6523 | 0.9369 | 0.5743 | 0.6536 | 0.6114 | 0.4074 | 0.3438 | 0.3729 | 0.8798 | 0.8944 | 0.8871 | 0.3333 | 0.375 | 0.3529 | 0.5769 | 0.5172 | 0.5455 | 0.5539 | 0.6533 | | 0.0031 | 28.0 | 3500 | 0.4100 | 0.6305 | 0.6755 | 0.6522 | 0.9351 | 0.5762 | 0.6392 | 0.6061 | 0.4074 | 0.3438 | 0.3729 | 0.8962 | 0.9111 | 0.9036 | 0.3421 | 0.4062 | 0.3714 | 0.5385 | 0.4828 | 0.5091 | 0.5526 | 0.6533 | | 0.0031 | 29.0 | 3625 | 0.4050 | 0.6383 | 0.6939 | 0.6650 | 0.9388 | 0.5916 | 0.6660 | 0.6266 | 0.44 | 0.3438 | 0.3860 | 0.8913 | 0.9111 | 0.9011 | 0.3095 | 0.4062 | 0.3514 | 0.5556 | 0.5172 | 0.5357 | 0.5601 | 0.6665 | | 0.0031 | 30.0 | 3750 | 0.4111 | 0.6348 | 0.6741 | 0.6539 | 0.9367 | 0.5819 | 0.6371 | 0.6083 | 0.4138 | 0.375 | 0.3934 | 0.8962 | 0.9111 | 0.9036 | 0.3243 | 0.375 | 0.3478 | 0.56 | 0.4828 | 0.5185 | 0.5543 | 0.6549 | | 0.0031 | 31.0 | 3875 | 0.4074 | 0.6349 | 0.6768 | 0.6552 | 0.9381 | 0.5832 | 0.6433 | 0.6118 | 0.3846 | 0.3125 | 0.3448 | 0.8962 | 0.9111 | 0.9036 | 0.3333 | 0.4062 | 0.3662 | 0.56 | 0.4828 | 0.5185 | 0.5490 | 0.6559 | | 0.002 | 32.0 | 4000 | 0.4086 | 0.6421 | 0.6794 | 0.6603 | 0.9379 | 0.5843 | 0.6433 | 0.6124 | 0.4138 | 0.375 | 0.3934 | 0.9016 | 0.9167 | 0.9091 | 0.375 | 0.375 | 0.375 | 0.5833 | 0.4828 | 0.5283 | 0.5636 | 0.6603 | | 0.002 | 33.0 | 4125 | 0.4174 | 0.6378 | 0.6900 | 0.6629 | 0.9369 | 0.5847 | 0.6619 | 0.6209 | 0.4074 | 0.3438 | 0.3729 | 0.9022 | 0.9222 | 0.9121 | 0.3235 | 0.3438 | 0.3333 | 0.5385 | 0.4828 | 0.5091 | 0.5497 | 0.6632 | | 0.002 | 34.0 | 4250 | 0.4131 | 0.6431 | 0.6847 | 0.6633 | 0.9379 | 0.5881 | 0.6536 | 0.6191 | 0.4138 | 0.375 | 0.3934 | 0.8962 | 0.9111 | 0.9036 | 0.375 | 0.375 | 0.375 | 0.5833 | 0.4828 | 0.5283 | 0.5639 | 0.6634 | | 0.002 | 35.0 | 4375 | 0.4142 | 0.6452 | 0.6860 | 0.6650 | 0.9380 | 0.5911 | 0.6557 | 0.6217 | 0.4138 | 0.375 | 0.3934 | 0.8962 | 0.9111 | 0.9036 | 0.375 | 0.375 | 0.375 | 0.5833 | 0.4828 | 0.5283 | 0.5644 | 0.6650 | ### Framework versions - Transformers 4.35.2 - Pytorch 2.1.0+cu121 - Datasets 2.15.0 - Tokenizers 0.15.0