---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: barthez-deft-sciences_de_l_information
  results:
  - task:
      name: Summarization
      type: summarization
    metrics:
    - name: Rouge1
      type: rouge
      value: 34.5672
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# barthez-deft-sciences_de_l_information

This model is a fine-tuned version of [moussaKam/barthez](https://huggingface.co/moussaKam/barthez) on an unknown dataset.

**Note**: this model is one of the preliminary experiments and it slightly underperforms the models published in the paper (using [MBartHez](https://huggingface.co/moussaKam/mbarthez) and HAL/Wiki pre-training + copy mechanisms)

It achieves the following results on the evaluation set:
- Loss: 2.0258
- Rouge1: 34.5672
- Rouge2: 16.7861
- Rougel: 27.5573
- Rougelsum: 27.6099
- Gen Len: 17.8857

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20.0
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| 3.3405        | 1.0   | 106  | 2.3682          | 31.3511 | 12.1973 | 25.6977 | 25.6851   | 14.9714 |
| 2.4219        | 2.0   | 212  | 2.1891          | 30.1154 | 13.3459 | 25.4854 | 25.5403   | 14.0429 |
| 2.0789        | 3.0   | 318  | 2.0994          | 32.153  | 15.3865 | 26.1859 | 26.1672   | 15.2    |
| 1.869         | 4.0   | 424  | 2.0258          | 34.5797 | 16.4194 | 27.6909 | 27.7201   | 16.9857 |
| 1.6569        | 5.0   | 530  | 2.0417          | 34.3854 | 16.5237 | 28.7036 | 28.8258   | 15.2429 |
| 1.5414        | 6.0   | 636  | 2.0503          | 33.1768 | 15.4851 | 27.2818 | 27.2884   | 16.0143 |
| 1.4461        | 7.0   | 742  | 2.0293          | 35.4273 | 16.118  | 27.3622 | 27.393    | 16.6857 |
| 1.3435        | 8.0   | 848  | 2.0336          | 35.3471 | 15.9695 | 27.668  | 27.6749   | 17.2    |
| 1.2624        | 9.0   | 954  | 2.0779          | 35.9201 | 17.2547 | 27.409  | 27.3293   | 17.1857 |
| 1.1807        | 10.0  | 1060 | 2.1301          | 35.7061 | 15.9138 | 27.3968 | 27.4716   | 17.1286 |
| 1.0972        | 11.0  | 1166 | 2.1726          | 34.3194 | 16.1313 | 27.0367 | 27.0737   | 17.1429 |
| 1.0224        | 12.0  | 1272 | 2.1704          | 34.9278 | 16.7958 | 27.8754 | 27.932    | 16.6571 |
| 1.0181        | 13.0  | 1378 | 2.2458          | 34.472  | 15.9111 | 28.2938 | 28.2946   | 16.7571 |
| 0.9769        | 14.0  | 1484 | 2.3405          | 35.1592 | 16.3135 | 29.0956 | 29.0858   | 16.5429 |
| 0.8866        | 15.0  | 1590 | 2.3303          | 34.8732 | 15.6709 | 27.5858 | 27.6169   | 16.2429 |
| 0.8888        | 16.0  | 1696 | 2.2976          | 35.3034 | 16.8011 | 27.7988 | 27.7569   | 17.5143 |
| 0.8358        | 17.0  | 1802 | 2.3349          | 35.505  | 16.8851 | 28.3651 | 28.413    | 16.8143 |
| 0.8026        | 18.0  | 1908 | 2.3738          | 35.2328 | 17.0358 | 28.544  | 28.6211   | 16.6143 |
| 0.7487        | 19.0  | 2014 | 2.4103          | 34.0793 | 15.4468 | 27.8057 | 27.8586   | 16.7286 |
| 0.7722        | 20.0  | 2120 | 2.3991          | 34.8116 | 15.8706 | 27.9173 | 27.983    | 16.9286 |


### Framework versions

- Transformers 4.10.2
- Pytorch 1.7.1+cu110
- Datasets 1.11.0
- Tokenizers 0.10.3