|
--- |
|
license: apache-2.0 |
|
base_model: facebook/bart-large |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- clupubhealth |
|
metrics: |
|
- rouge |
|
model-index: |
|
- name: bart-pubhealth-expanded |
|
results: |
|
- task: |
|
name: Sequence-to-sequence Language Modeling |
|
type: text2text-generation |
|
dataset: |
|
name: clupubhealth |
|
type: clupubhealth |
|
config: expanded |
|
split: test |
|
args: expanded |
|
metrics: |
|
- name: Rouge1 |
|
type: rouge |
|
value: 29.8528 |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# bart-pubhealth-expanded |
|
|
|
This model is a fine-tuned version of [facebook/bart-large](https://huggingface.co/facebook/bart-large) on the clupubhealth dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 2.3926 |
|
- Rouge1: 29.8528 |
|
- Rouge2: 10.8495 |
|
- Rougel: 23.3682 |
|
- Rougelsum: 23.7565 |
|
- Gen Len: 19.85 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 2e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- gradient_accumulation_steps: 2 |
|
- total_train_batch_size: 32 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 10 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len | |
|
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:| |
|
| 2.7469 | 0.26 | 500 | 2.0845 | 30.9611 | 10.7145 | 23.9719 | 24.1042 | 19.905 | |
|
| 2.5524 | 0.51 | 1000 | 2.0628 | 32.0352 | 11.8898 | 24.9032 | 25.1368 | 19.895 | |
|
| 2.429 | 0.77 | 1500 | 2.0787 | 32.2632 | 12.0353 | 25.1245 | 25.3728 | 19.895 | |
|
| 2.2234 | 1.03 | 2000 | 2.1178 | 30.6437 | 11.5713 | 24.9071 | 25.1126 | 19.955 | |
|
| 2.1249 | 1.29 | 2500 | 2.1183 | 31.6095 | 11.6573 | 25.0593 | 25.2063 | 19.87 | |
|
| 2.0302 | 1.54 | 3000 | 2.1319 | 30.7417 | 11.4924 | 24.6388 | 24.8722 | 19.895 | |
|
| 1.9761 | 1.8 | 3500 | 2.1850 | 31.6709 | 11.3036 | 24.4853 | 24.7571 | 19.87 | |
|
| 1.8279 | 2.06 | 4000 | 2.2092 | 31.5778 | 11.59 | 24.7599 | 24.9956 | 19.825 | |
|
| 1.8083 | 2.32 | 4500 | 2.1781 | 31.0441 | 10.7513 | 24.0656 | 24.3112 | 19.89 | |
|
| 1.7527 | 2.57 | 5000 | 2.2155 | 31.1191 | 11.4519 | 24.4673 | 24.7157 | 19.81 | |
|
| 1.723 | 2.83 | 5500 | 2.2024 | 31.9787 | 12.3158 | 24.9863 | 25.2597 | 19.94 | |
|
| 1.5975 | 3.09 | 6000 | 2.2567 | 31.236 | 10.9733 | 24.1302 | 24.3433 | 19.9 | |
|
| 1.5933 | 3.35 | 6500 | 2.2425 | 31.022 | 11.0249 | 24.1257 | 24.3555 | 19.92 | |
|
| 1.5792 | 3.6 | 7000 | 2.2428 | 29.8844 | 10.3622 | 23.0802 | 23.4003 | 19.96 | |
|
| 1.5718 | 3.86 | 7500 | 2.2367 | 31.2369 | 11.3854 | 24.8528 | 25.1287 | 19.815 | |
|
| 1.4467 | 4.12 | 8000 | 2.2988 | 30.4903 | 10.4057 | 23.9914 | 24.239 | 19.715 | |
|
| 1.4458 | 4.37 | 8500 | 2.2738 | 31.4345 | 11.2989 | 24.4239 | 24.6047 | 19.75 | |
|
| 1.4342 | 4.63 | 9000 | 2.3092 | 28.8421 | 10.5744 | 23.0084 | 23.1741 | 19.855 | |
|
| 1.4416 | 4.89 | 9500 | 2.2747 | 31.7111 | 11.5903 | 24.3422 | 24.6867 | 19.945 | |
|
| 1.3437 | 5.15 | 10000 | 2.3203 | 31.11 | 11.0 | 24.6098 | 24.7362 | 19.81 | |
|
| 1.3525 | 5.4 | 10500 | 2.3085 | 29.414 | 10.3412 | 23.3134 | 23.6552 | 19.935 | |
|
| 1.3533 | 5.66 | 11000 | 2.3123 | 31.321 | 11.2686 | 23.9922 | 24.336 | 19.77 | |
|
| 1.3248 | 5.92 | 11500 | 2.2916 | 30.8841 | 10.779 | 23.9407 | 24.0865 | 19.845 | |
|
| 1.2617 | 6.18 | 12000 | 2.3530 | 29.7167 | 10.3162 | 23.4805 | 23.724 | 19.93 | |
|
| 1.2846 | 6.43 | 12500 | 2.3712 | 28.3334 | 9.8425 | 22.1151 | 22.2951 | 19.92 | |
|
| 1.2472 | 6.69 | 13000 | 2.3378 | 29.563 | 10.0033 | 23.1863 | 23.5065 | 19.865 | |
|
| 1.2934 | 6.95 | 13500 | 2.3262 | 29.137 | 10.1232 | 22.9234 | 23.3799 | 19.855 | |
|
| 1.2136 | 7.21 | 14000 | 2.3640 | 29.753 | 10.4865 | 23.4892 | 23.8778 | 19.885 | |
|
| 1.2096 | 7.46 | 14500 | 2.3654 | 29.512 | 10.3891 | 23.0427 | 23.3684 | 19.88 | |
|
| 1.211 | 7.72 | 15000 | 2.3491 | 30.9014 | 10.9117 | 24.127 | 24.3518 | 19.785 | |
|
| 1.1954 | 7.98 | 15500 | 2.3626 | 29.0622 | 10.5405 | 22.7407 | 22.9454 | 19.84 | |
|
| 1.1756 | 8.23 | 16000 | 2.3759 | 29.5277 | 10.2961 | 22.7888 | 23.1239 | 19.88 | |
|
| 1.1516 | 8.49 | 16500 | 2.3772 | 29.3161 | 10.1894 | 23.0404 | 23.486 | 19.885 | |
|
| 1.1604 | 8.75 | 17000 | 2.3710 | 29.6161 | 10.3543 | 22.8748 | 23.1849 | 19.905 | |
|
| 1.1639 | 9.01 | 17500 | 2.3889 | 30.2817 | 10.8654 | 23.6438 | 23.8639 | 19.895 | |
|
| 1.12 | 9.26 | 18000 | 2.3968 | 28.8747 | 9.8686 | 22.2775 | 22.6541 | 19.895 | |
|
| 1.1136 | 9.52 | 18500 | 2.3950 | 30.1197 | 10.8992 | 23.2575 | 23.5732 | 19.86 | |
|
| 1.1437 | 9.78 | 19000 | 2.3926 | 29.8528 | 10.8495 | 23.3682 | 23.7565 | 19.85 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.31.0 |
|
- Pytorch 2.0.1+cu117 |
|
- Datasets 2.7.1 |
|
- Tokenizers 0.13.2 |
|
|
