Training complete

README.md

@@ -1,6 +1,7 @@
 ---
 library_name: transformers
-base_model: allenai/biomed_roberta_base
+license: mit
+base_model: emilyalsentzer/Bio_ClinicalBERT
 tags:
 - generated_from_trainer
 metrics:
@@ -18,13 +19,13 @@ should probably proofread and complete it, then remove this comment. -->
 
 # BioMedRoBERTa-finetuned-ner-pablo-just-classifier
 
-This model is a fine-tuned version of [allenai/biomed_roberta_base](https://huggingface.co/allenai/biomed_roberta_base) on the None dataset.
+This model is a fine-tuned version of [emilyalsentzer/Bio_ClinicalBERT](https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.1276
-- Precision: 0.6818
-- Recall: 0.7031
-- F1: 0.6923
-- Accuracy: 0.9672
+- Loss: 0.1150
+- Precision: 0.6869
+- Recall: 0.7076
+- F1: 0.6971
+- Accuracy: 0.9677
 
 ## Model description
 
@@ -44,24 +45,31 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.1
-- train_batch_size: 2
-- eval_batch_size: 2
+- train_batch_size: 128
+- eval_batch_size: 128
 - seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 512
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
+- lr_scheduler_type: cosine_with_restarts
 - lr_scheduler_warmup_ratio: 0.05
-- num_epochs: 5
+- num_epochs: 10
 - mixed_precision_training: Native AMP
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
-|:-------------:|:-----:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
-| 0.7191 | 1.0 | 2509 | 0.5248 | 0.5127 | 0.6334 | 0.5667 | 0.9486 |
-| 0.5382 | 2.0 | 5018 | 0.4280 | 0.5378 | 0.6500 | 0.5886 | 0.9556 |
-| 0.3968 | 3.0 | 7527 | 0.3095 | 0.4997 | 0.6714 | 0.5730 | 0.9531 |
-| 0.2528 | 4.0 | 10036 | 0.1872 | 0.5631 | 0.6850 | 0.6181 | 0.9599 |
-| 0.1541 | 5.0 | 12545 | 0.1276 | 0.6818 | 0.7031 | 0.6923 | 0.9672 |
+| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
+|:-------------:|:------:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
+| No log | 0.9655 | 14 | 0.3729 | 0.4205 | 0.6119 | 0.4985 | 0.9430 |
+| No log | 2.0 | 29 | 0.2544 | 0.5272 | 0.6683 | 0.5894 | 0.9574 |
+| No log | 2.9655 | 43 | 0.2117 | 0.5702 | 0.6884 | 0.6238 | 0.9604 |
+| No log | 4.0 | 58 | 0.1747 | 0.5934 | 0.7001 | 0.6424 | 0.9628 |
+| No log | 4.9655 | 72 | 0.1420 | 0.6280 | 0.6827 | 0.6542 | 0.9642 |
+| No log | 6.0 | 87 | 0.1287 | 0.6639 | 0.7033 | 0.6830 | 0.9667 |
+| No log | 6.9655 | 101 | 0.1309 | 0.6471 | 0.7009 | 0.6729 | 0.9654 |
+| No log | 8.0 | 116 | 0.1260 | 0.6349 | 0.7199 | 0.6748 | 0.9652 |
+| No log | 8.9655 | 130 | 0.1159 | 0.6621 | 0.7118 | 0.6860 | 0.9670 |
+| No log | 9.6552 | 140 | 0.1150 | 0.6869 | 0.7076 | 0.6971 | 0.9677 |
 
 
 ### Framework versions