Training complete
Browse files
README.md
ADDED
|
@@ -0,0 +1,117 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
base_model: roberta-base
|
| 4 |
+
tags:
|
| 5 |
+
- generated_from_trainer
|
| 6 |
+
metrics:
|
| 7 |
+
- precision
|
| 8 |
+
- recall
|
| 9 |
+
- f1
|
| 10 |
+
- accuracy
|
| 11 |
+
model-index:
|
| 12 |
+
- name: roberta-base-finetuned-ner-cadec-no-iob
|
| 13 |
+
results: []
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
| 17 |
+
should probably proofread and complete it, then remove this comment. -->
|
| 18 |
+
|
| 19 |
+
# roberta-base-finetuned-ner-cadec-no-iob
|
| 20 |
+
|
| 21 |
+
This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
|
| 22 |
+
It achieves the following results on the evaluation set:
|
| 23 |
+
- Loss: 0.4142
|
| 24 |
+
- Precision: 0.6452
|
| 25 |
+
- Recall: 0.6860
|
| 26 |
+
- F1: 0.6650
|
| 27 |
+
- Accuracy: 0.9380
|
| 28 |
+
- Adr Precision: 0.5911
|
| 29 |
+
- Adr Recall: 0.6557
|
| 30 |
+
- Adr F1: 0.6217
|
| 31 |
+
- Disease Precision: 0.4138
|
| 32 |
+
- Disease Recall: 0.375
|
| 33 |
+
- Disease F1: 0.3934
|
| 34 |
+
- Drug Precision: 0.8962
|
| 35 |
+
- Drug Recall: 0.9111
|
| 36 |
+
- Drug F1: 0.9036
|
| 37 |
+
- Finding Precision: 0.375
|
| 38 |
+
- Finding Recall: 0.375
|
| 39 |
+
- Finding F1: 0.375
|
| 40 |
+
- Symptom Precision: 0.5833
|
| 41 |
+
- Symptom Recall: 0.4828
|
| 42 |
+
- Symptom F1: 0.5283
|
| 43 |
+
- Macro Avg F1: 0.5644
|
| 44 |
+
- Weighted Avg F1: 0.6650
|
| 45 |
+
|
| 46 |
+
## Model description
|
| 47 |
+
|
| 48 |
+
More information needed
|
| 49 |
+
|
| 50 |
+
## Intended uses & limitations
|
| 51 |
+
|
| 52 |
+
More information needed
|
| 53 |
+
|
| 54 |
+
## Training and evaluation data
|
| 55 |
+
|
| 56 |
+
More information needed
|
| 57 |
+
|
| 58 |
+
## Training procedure
|
| 59 |
+
|
| 60 |
+
### Training hyperparameters
|
| 61 |
+
|
| 62 |
+
The following hyperparameters were used during training:
|
| 63 |
+
- learning_rate: 2e-05
|
| 64 |
+
- train_batch_size: 8
|
| 65 |
+
- eval_batch_size: 8
|
| 66 |
+
- seed: 42
|
| 67 |
+
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
| 68 |
+
- lr_scheduler_type: linear
|
| 69 |
+
- num_epochs: 35
|
| 70 |
+
|
| 71 |
+
### Training results
|
| 72 |
+
|
| 73 |
+
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy | Adr Precision | Adr Recall | Adr F1 | Disease Precision | Disease Recall | Disease F1 | Drug Precision | Drug Recall | Drug F1 | Finding Precision | Finding Recall | Finding F1 | Symptom Precision | Symptom Recall | Symptom F1 | Macro Avg F1 | Weighted Avg F1 |
|
| 74 |
+
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|:-------------:|:----------:|:------:|:-----------------:|:--------------:|:----------:|:--------------:|:-----------:|:-------:|:-----------------:|:--------------:|:----------:|:-----------------:|:--------------:|:----------:|:------------:|:---------------:|
|
| 75 |
+
| No log | 1.0 | 125 | 0.2142 | 0.5325 | 0.6055 | 0.5667 | 0.9194 | 0.4548 | 0.5918 | 0.5143 | 0.4186 | 0.5625 | 0.48 | 0.8398 | 0.8444 | 0.8421 | 0.2857 | 0.0625 | 0.1026 | 0.0 | 0.0 | 0.0 | 0.3878 | 0.5537 |
|
| 76 |
+
| No log | 2.0 | 250 | 0.1798 | 0.6083 | 0.6557 | 0.6311 | 0.9339 | 0.5276 | 0.6309 | 0.5746 | 0.6842 | 0.4062 | 0.5098 | 0.8950 | 0.9 | 0.8975 | 0.32 | 0.25 | 0.2807 | 0.6667 | 0.2759 | 0.3902 | 0.5306 | 0.6291 |
|
| 77 |
+
| No log | 3.0 | 375 | 0.1910 | 0.5748 | 0.6029 | 0.5885 | 0.9282 | 0.5191 | 0.5320 | 0.5255 | 0.6 | 0.375 | 0.4615 | 0.8820 | 0.8722 | 0.8771 | 0.2927 | 0.375 | 0.3288 | 0.3051 | 0.6207 | 0.4091 | 0.5204 | 0.5935 |
|
| 78 |
+
| 0.1902 | 4.0 | 500 | 0.2013 | 0.5995 | 0.6398 | 0.6190 | 0.9311 | 0.5460 | 0.6 | 0.5717 | 0.25 | 0.0938 | 0.1364 | 0.8840 | 0.8889 | 0.8864 | 0.2632 | 0.4688 | 0.3371 | 0.6154 | 0.5517 | 0.5818 | 0.5027 | 0.6185 |
|
| 79 |
+
| 0.1902 | 5.0 | 625 | 0.2113 | 0.6161 | 0.6649 | 0.6396 | 0.9335 | 0.5515 | 0.6289 | 0.5877 | 0.5556 | 0.4688 | 0.5085 | 0.8852 | 0.9 | 0.8926 | 0.2857 | 0.25 | 0.2667 | 0.5185 | 0.4828 | 0.5 | 0.5511 | 0.6398 |
|
| 80 |
+
| 0.1902 | 6.0 | 750 | 0.1955 | 0.6223 | 0.6544 | 0.6379 | 0.9341 | 0.5541 | 0.6021 | 0.5771 | 0.5833 | 0.4375 | 0.5 | 0.8956 | 0.9056 | 0.9006 | 0.2857 | 0.375 | 0.3243 | 0.6818 | 0.5172 | 0.5882 | 0.5780 | 0.6404 |
|
| 81 |
+
| 0.1902 | 7.0 | 875 | 0.2226 | 0.6252 | 0.6491 | 0.6369 | 0.9343 | 0.5671 | 0.6186 | 0.5917 | 0.5556 | 0.1562 | 0.2439 | 0.8983 | 0.8833 | 0.8908 | 0.3061 | 0.4688 | 0.3704 | 0.5652 | 0.4483 | 0.5000 | 0.5193 | 0.6352 |
|
| 82 |
+
| 0.0648 | 8.0 | 1000 | 0.2345 | 0.6229 | 0.6755 | 0.6481 | 0.9363 | 0.5773 | 0.6392 | 0.6067 | 0.4138 | 0.375 | 0.3934 | 0.875 | 0.8944 | 0.8846 | 0.2973 | 0.3438 | 0.3188 | 0.5143 | 0.6207 | 0.5625 | 0.5532 | 0.6498 |
|
| 83 |
+
| 0.0648 | 9.0 | 1125 | 0.2316 | 0.6322 | 0.6689 | 0.65 | 0.9368 | 0.5851 | 0.6309 | 0.6071 | 0.5 | 0.3125 | 0.3846 | 0.8811 | 0.9056 | 0.8932 | 0.2766 | 0.4062 | 0.3291 | 0.5556 | 0.5172 | 0.5357 | 0.5499 | 0.6512 |
|
| 84 |
+
| 0.0648 | 10.0 | 1250 | 0.2944 | 0.6204 | 0.6491 | 0.6344 | 0.9320 | 0.5551 | 0.6021 | 0.5776 | 0.5789 | 0.3438 | 0.4314 | 0.8913 | 0.9111 | 0.9011 | 0.2619 | 0.3438 | 0.2973 | 0.6364 | 0.4828 | 0.5490 | 0.5513 | 0.6353 |
|
| 85 |
+
| 0.0648 | 11.0 | 1375 | 0.2660 | 0.6280 | 0.6794 | 0.6527 | 0.9353 | 0.5786 | 0.6454 | 0.6101 | 0.3824 | 0.4062 | 0.3939 | 0.8956 | 0.9056 | 0.9006 | 0.2812 | 0.2812 | 0.2812 | 0.5484 | 0.5862 | 0.5667 | 0.5505 | 0.6544 |
|
| 86 |
+
| 0.0284 | 12.0 | 1500 | 0.2819 | 0.6366 | 0.6702 | 0.6530 | 0.9355 | 0.5827 | 0.6392 | 0.6096 | 0.5 | 0.1875 | 0.2727 | 0.8956 | 0.9056 | 0.9006 | 0.3478 | 0.5 | 0.4103 | 0.5 | 0.4483 | 0.4727 | 0.5332 | 0.6508 |
|
| 87 |
+
| 0.0284 | 13.0 | 1625 | 0.3000 | 0.6326 | 0.6702 | 0.6509 | 0.9363 | 0.5736 | 0.6351 | 0.6027 | 0.4444 | 0.375 | 0.4068 | 0.8950 | 0.9 | 0.8975 | 0.3421 | 0.4062 | 0.3714 | 0.65 | 0.4483 | 0.5306 | 0.5618 | 0.6519 |
|
| 88 |
+
| 0.0284 | 14.0 | 1750 | 0.2996 | 0.6228 | 0.6491 | 0.6357 | 0.9363 | 0.5645 | 0.6041 | 0.5837 | 0.4783 | 0.3438 | 0.4 | 0.8743 | 0.8889 | 0.8815 | 0.2973 | 0.3438 | 0.3188 | 0.6071 | 0.5862 | 0.5965 | 0.5561 | 0.6360 |
|
| 89 |
+
| 0.0284 | 15.0 | 1875 | 0.3246 | 0.6311 | 0.6636 | 0.6469 | 0.9352 | 0.5788 | 0.6206 | 0.5990 | 0.45 | 0.2812 | 0.3462 | 0.9056 | 0.9056 | 0.9056 | 0.2683 | 0.3438 | 0.3014 | 0.5278 | 0.6552 | 0.5846 | 0.5473 | 0.6480 |
|
| 90 |
+
| 0.0136 | 16.0 | 2000 | 0.3305 | 0.6461 | 0.6623 | 0.6541 | 0.9377 | 0.5869 | 0.6268 | 0.6062 | 0.4545 | 0.3125 | 0.3704 | 0.9011 | 0.9111 | 0.9061 | 0.3448 | 0.3125 | 0.3279 | 0.5385 | 0.4828 | 0.5091 | 0.5439 | 0.6520 |
|
| 91 |
+
| 0.0136 | 17.0 | 2125 | 0.3181 | 0.6291 | 0.6781 | 0.6527 | 0.9375 | 0.5780 | 0.6495 | 0.6117 | 0.4231 | 0.3438 | 0.3793 | 0.9066 | 0.9167 | 0.9116 | 0.2857 | 0.3125 | 0.2985 | 0.4483 | 0.4483 | 0.4483 | 0.5299 | 0.6536 |
|
| 92 |
+
| 0.0136 | 18.0 | 2250 | 0.3414 | 0.6298 | 0.6755 | 0.6518 | 0.9362 | 0.5765 | 0.6371 | 0.6053 | 0.375 | 0.375 | 0.375 | 0.8962 | 0.9111 | 0.9036 | 0.3235 | 0.3438 | 0.3333 | 0.5714 | 0.5517 | 0.5614 | 0.5557 | 0.6532 |
|
| 93 |
+
| 0.0136 | 19.0 | 2375 | 0.3457 | 0.6302 | 0.6768 | 0.6527 | 0.9372 | 0.5877 | 0.6495 | 0.6170 | 0.3636 | 0.25 | 0.2963 | 0.8907 | 0.9056 | 0.8981 | 0.26 | 0.4062 | 0.3171 | 0.6087 | 0.4828 | 0.5385 | 0.5334 | 0.6546 |
|
| 94 |
+
| 0.0078 | 20.0 | 2500 | 0.3700 | 0.6367 | 0.6636 | 0.6499 | 0.9367 | 0.5805 | 0.6247 | 0.6018 | 0.3714 | 0.4062 | 0.3881 | 0.9016 | 0.9167 | 0.9091 | 0.3077 | 0.25 | 0.2759 | 0.5833 | 0.4828 | 0.5283 | 0.5406 | 0.6492 |
|
| 95 |
+
| 0.0078 | 21.0 | 2625 | 0.3772 | 0.6276 | 0.6715 | 0.6488 | 0.9325 | 0.5766 | 0.6289 | 0.6016 | 0.44 | 0.3438 | 0.3860 | 0.8919 | 0.9167 | 0.9041 | 0.2927 | 0.375 | 0.3288 | 0.5161 | 0.5517 | 0.5333 | 0.5508 | 0.6502 |
|
| 96 |
+
| 0.0078 | 22.0 | 2750 | 0.3622 | 0.6389 | 0.6768 | 0.6573 | 0.9345 | 0.5855 | 0.6495 | 0.6158 | 0.4333 | 0.4062 | 0.4194 | 0.8840 | 0.8889 | 0.8864 | 0.3333 | 0.3125 | 0.3226 | 0.625 | 0.5172 | 0.5660 | 0.5620 | 0.6575 |
|
| 97 |
+
| 0.0078 | 23.0 | 2875 | 0.3811 | 0.6304 | 0.6728 | 0.6509 | 0.9352 | 0.5765 | 0.6371 | 0.6053 | 0.4 | 0.375 | 0.3871 | 0.8804 | 0.9 | 0.8901 | 0.3438 | 0.3438 | 0.3438 | 0.5926 | 0.5517 | 0.5714 | 0.5595 | 0.6514 |
|
| 98 |
+
| 0.005 | 24.0 | 3000 | 0.3824 | 0.6322 | 0.6689 | 0.65 | 0.9353 | 0.5757 | 0.6351 | 0.6039 | 0.4286 | 0.375 | 0.4000 | 0.8901 | 0.9 | 0.8950 | 0.3226 | 0.3125 | 0.3175 | 0.5769 | 0.5172 | 0.5455 | 0.5524 | 0.6501 |
|
| 99 |
+
| 0.005 | 25.0 | 3125 | 0.3821 | 0.6297 | 0.6821 | 0.6548 | 0.9375 | 0.5850 | 0.6598 | 0.6202 | 0.4 | 0.375 | 0.3871 | 0.8852 | 0.9 | 0.8926 | 0.25 | 0.2812 | 0.2647 | 0.56 | 0.4828 | 0.5185 | 0.5366 | 0.6561 |
|
| 100 |
+
| 0.005 | 26.0 | 3250 | 0.4058 | 0.6292 | 0.6715 | 0.6496 | 0.9355 | 0.5821 | 0.6433 | 0.6112 | 0.3939 | 0.4062 | 0.4 | 0.875 | 0.8944 | 0.8846 | 0.2857 | 0.25 | 0.2667 | 0.5357 | 0.5172 | 0.5263 | 0.5378 | 0.6494 |
|
| 101 |
+
| 0.005 | 27.0 | 3375 | 0.3980 | 0.6262 | 0.6807 | 0.6523 | 0.9369 | 0.5743 | 0.6536 | 0.6114 | 0.4074 | 0.3438 | 0.3729 | 0.8798 | 0.8944 | 0.8871 | 0.3333 | 0.375 | 0.3529 | 0.5769 | 0.5172 | 0.5455 | 0.5539 | 0.6533 |
|
| 102 |
+
| 0.0031 | 28.0 | 3500 | 0.4100 | 0.6305 | 0.6755 | 0.6522 | 0.9351 | 0.5762 | 0.6392 | 0.6061 | 0.4074 | 0.3438 | 0.3729 | 0.8962 | 0.9111 | 0.9036 | 0.3421 | 0.4062 | 0.3714 | 0.5385 | 0.4828 | 0.5091 | 0.5526 | 0.6533 |
|
| 103 |
+
| 0.0031 | 29.0 | 3625 | 0.4050 | 0.6383 | 0.6939 | 0.6650 | 0.9388 | 0.5916 | 0.6660 | 0.6266 | 0.44 | 0.3438 | 0.3860 | 0.8913 | 0.9111 | 0.9011 | 0.3095 | 0.4062 | 0.3514 | 0.5556 | 0.5172 | 0.5357 | 0.5601 | 0.6665 |
|
| 104 |
+
| 0.0031 | 30.0 | 3750 | 0.4111 | 0.6348 | 0.6741 | 0.6539 | 0.9367 | 0.5819 | 0.6371 | 0.6083 | 0.4138 | 0.375 | 0.3934 | 0.8962 | 0.9111 | 0.9036 | 0.3243 | 0.375 | 0.3478 | 0.56 | 0.4828 | 0.5185 | 0.5543 | 0.6549 |
|
| 105 |
+
| 0.0031 | 31.0 | 3875 | 0.4074 | 0.6349 | 0.6768 | 0.6552 | 0.9381 | 0.5832 | 0.6433 | 0.6118 | 0.3846 | 0.3125 | 0.3448 | 0.8962 | 0.9111 | 0.9036 | 0.3333 | 0.4062 | 0.3662 | 0.56 | 0.4828 | 0.5185 | 0.5490 | 0.6559 |
|
| 106 |
+
| 0.002 | 32.0 | 4000 | 0.4086 | 0.6421 | 0.6794 | 0.6603 | 0.9379 | 0.5843 | 0.6433 | 0.6124 | 0.4138 | 0.375 | 0.3934 | 0.9016 | 0.9167 | 0.9091 | 0.375 | 0.375 | 0.375 | 0.5833 | 0.4828 | 0.5283 | 0.5636 | 0.6603 |
|
| 107 |
+
| 0.002 | 33.0 | 4125 | 0.4174 | 0.6378 | 0.6900 | 0.6629 | 0.9369 | 0.5847 | 0.6619 | 0.6209 | 0.4074 | 0.3438 | 0.3729 | 0.9022 | 0.9222 | 0.9121 | 0.3235 | 0.3438 | 0.3333 | 0.5385 | 0.4828 | 0.5091 | 0.5497 | 0.6632 |
|
| 108 |
+
| 0.002 | 34.0 | 4250 | 0.4131 | 0.6431 | 0.6847 | 0.6633 | 0.9379 | 0.5881 | 0.6536 | 0.6191 | 0.4138 | 0.375 | 0.3934 | 0.8962 | 0.9111 | 0.9036 | 0.375 | 0.375 | 0.375 | 0.5833 | 0.4828 | 0.5283 | 0.5639 | 0.6634 |
|
| 109 |
+
| 0.002 | 35.0 | 4375 | 0.4142 | 0.6452 | 0.6860 | 0.6650 | 0.9380 | 0.5911 | 0.6557 | 0.6217 | 0.4138 | 0.375 | 0.3934 | 0.8962 | 0.9111 | 0.9036 | 0.375 | 0.375 | 0.375 | 0.5833 | 0.4828 | 0.5283 | 0.5644 | 0.6650 |
|
| 110 |
+
|
| 111 |
+
|
| 112 |
+
### Framework versions
|
| 113 |
+
|
| 114 |
+
- Transformers 4.35.2
|
| 115 |
+
- Pytorch 2.1.0+cu121
|
| 116 |
+
- Datasets 2.15.0
|
| 117 |
+
- Tokenizers 0.15.0
|