---
license: mit
base_model: FacebookAI/xlm-roberta-large
tags:
- generated_from_trainer
datasets:
- cnec
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: CNEC1_1_Supertypes_xlm-roberta-large
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: cnec
      type: cnec
      config: default
      split: validation
      args: default
    metrics:
    - name: Precision
      type: precision
      value: 0.829450915141431
    - name: Recall
      type: recall
      value: 0.8815207780725022
    - name: F1
      type: f1
      value: 0.8546935276468067
    - name: Accuracy
      type: accuracy
      value: 0.9615129396151294
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# CNEC1_1_Supertypes_xlm-roberta-large

This model is a fine-tuned version of [FacebookAI/xlm-roberta-large](https://huggingface.co/FacebookAI/xlm-roberta-large) on the cnec dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3089
- Precision: 0.8295
- Recall: 0.8815
- F1: 0.8547
- Accuracy: 0.9615

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.7892        | 0.11  | 500   | 0.5205          | 0.5042    | 0.6631 | 0.5728 | 0.9114   |
| 0.5593        | 0.21  | 1000  | 0.3722          | 0.6071    | 0.7241 | 0.6605 | 0.9246   |
| 0.5423        | 0.32  | 1500  | 0.3822          | 0.6001    | 0.7445 | 0.6646 | 0.9289   |
| 0.4507        | 0.43  | 2000  | 0.3354          | 0.7023    | 0.8073 | 0.7511 | 0.9423   |
| 0.4301        | 0.53  | 2500  | 0.3867          | 0.6740    | 0.7878 | 0.7265 | 0.9364   |
| 0.4034        | 0.64  | 3000  | 0.3318          | 0.7601    | 0.8011 | 0.7800 | 0.9435   |
| 0.4184        | 0.75  | 3500  | 0.3698          | 0.7133    | 0.7732 | 0.7420 | 0.9407   |
| 0.4236        | 0.85  | 4000  | 0.3648          | 0.7252    | 0.8258 | 0.7722 | 0.9415   |
| 0.3578        | 0.96  | 4500  | 0.2941          | 0.7341    | 0.8373 | 0.7823 | 0.9450   |
| 0.4098        | 1.06  | 5000  | 0.3095          | 0.7428    | 0.8386 | 0.7878 | 0.9456   |
| 0.3196        | 1.17  | 5500  | 0.2866          | 0.7563    | 0.8532 | 0.8018 | 0.9486   |
| 0.3572        | 1.28  | 6000  | 0.3165          | 0.7445    | 0.8232 | 0.7819 | 0.9479   |
| 0.2977        | 1.38  | 6500  | 0.2743          | 0.7762    | 0.8249 | 0.7998 | 0.9487   |
| 0.279         | 1.49  | 7000  | 0.3120          | 0.7637    | 0.8572 | 0.8077 | 0.9525   |
| 0.308         | 1.6   | 7500  | 0.3020          | 0.7589    | 0.8364 | 0.7958 | 0.9465   |
| 0.2879        | 1.7   | 8000  | 0.2717          | 0.7817    | 0.8594 | 0.8187 | 0.9514   |
| 0.2365        | 1.81  | 8500  | 0.2849          | 0.7780    | 0.8554 | 0.8149 | 0.9535   |
| 0.2658        | 1.92  | 9000  | 0.2476          | 0.7829    | 0.8484 | 0.8143 | 0.9551   |
| 0.2698        | 2.02  | 9500  | 0.3102          | 0.7622    | 0.8457 | 0.8018 | 0.9520   |
| 0.2297        | 2.13  | 10000 | 0.3206          | 0.7767    | 0.8612 | 0.8168 | 0.9523   |
| 0.2489        | 2.24  | 10500 | 0.3014          | 0.7662    | 0.8722 | 0.8158 | 0.9522   |
| 0.1962        | 2.34  | 11000 | 0.2986          | 0.7723    | 0.8652 | 0.8161 | 0.9528   |
| 0.2458        | 2.45  | 11500 | 0.2584          | 0.7812    | 0.8696 | 0.8230 | 0.9536   |
| 0.2108        | 2.56  | 12000 | 0.2866          | 0.7922    | 0.8798 | 0.8337 | 0.9575   |
| 0.2838        | 2.66  | 12500 | 0.2601          | 0.7740    | 0.8599 | 0.8147 | 0.9557   |
| 0.2092        | 2.77  | 13000 | 0.3028          | 0.7846    | 0.8585 | 0.8199 | 0.9536   |
| 0.2121        | 2.88  | 13500 | 0.2591          | 0.7798    | 0.8687 | 0.8218 | 0.9568   |
| 0.2445        | 2.98  | 14000 | 0.2742          | 0.7921    | 0.8758 | 0.8318 | 0.9557   |
| 0.1728        | 3.09  | 14500 | 0.2930          | 0.8080    | 0.8612 | 0.8337 | 0.9571   |
| 0.1426        | 3.19  | 15000 | 0.2775          | 0.7932    | 0.8665 | 0.8282 | 0.9569   |
| 0.1705        | 3.3   | 15500 | 0.3666          | 0.8082    | 0.8683 | 0.8372 | 0.9570   |
| 0.1324        | 3.41  | 16000 | 0.3011          | 0.8041    | 0.8727 | 0.8370 | 0.9574   |
| 0.1716        | 3.51  | 16500 | 0.3092          | 0.8002    | 0.8572 | 0.8277 | 0.9568   |
| 0.1465        | 3.62  | 17000 | 0.3439          | 0.7970    | 0.8643 | 0.8293 | 0.9574   |
| 0.1277        | 3.73  | 17500 | 0.3394          | 0.8126    | 0.8798 | 0.8448 | 0.9586   |
| 0.2132        | 3.83  | 18000 | 0.3296          | 0.8085    | 0.8736 | 0.8398 | 0.9589   |
| 0.211         | 3.94  | 18500 | 0.3000          | 0.8220    | 0.8820 | 0.8509 | 0.9587   |
| 0.137         | 4.05  | 19000 | 0.3241          | 0.8132    | 0.8740 | 0.8425 | 0.9582   |
| 0.1247        | 4.15  | 19500 | 0.3241          | 0.8205    | 0.8771 | 0.8479 | 0.9590   |
| 0.118         | 4.26  | 20000 | 0.3204          | 0.8336    | 0.8749 | 0.8538 | 0.9594   |
| 0.1403        | 4.37  | 20500 | 0.3130          | 0.8345    | 0.8780 | 0.8557 | 0.9612   |
| 0.1354        | 4.47  | 21000 | 0.3173          | 0.8229    | 0.8811 | 0.8510 | 0.9604   |
| 0.124         | 4.58  | 21500 | 0.3079          | 0.8236    | 0.8811 | 0.8513 | 0.9616   |
| 0.1115        | 4.69  | 22000 | 0.3020          | 0.8336    | 0.8815 | 0.8569 | 0.9618   |
| 0.0826        | 4.79  | 22500 | 0.3018          | 0.8301    | 0.8811 | 0.8548 | 0.9619   |
| 0.1467        | 4.9   | 23000 | 0.3089          | 0.8295    | 0.8815 | 0.8547 | 0.9615   |


### Framework versions

- Transformers 4.36.2
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0