checkpoints

This model is a fine-tuned version of nielsr/lilt-xlm-roberta-base on the xfun dataset. It achieves the following results on the evaluation set:

  • Precision: 0.3054
  • Recall: 0.6032
  • F1: 0.4055
  • Loss: 0.2164

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 8
  • eval_batch_size: 2
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • training_steps: 10000

Training results

Training Loss Epoch Step Precision Recall F1 Validation Loss
0.1914 20.83 500 0 0 0 0.2039
0.1638 41.67 1000 0.4688 0.0252 0.0478 0.2196
0.0928 62.5 1500 0.3790 0.1669 0.2318 0.2127
0.0948 83.33 2000 0.3125 0.4245 0.3600 0.2987
0.0796 104.17 2500 0.3102 0.5587 0.3989 0.3636
0.0469 125.0 3000 0.3204 0.5134 0.3946 0.3587
0.0471 145.83 3500 0.3303 0.5243 0.4053 0.2792
0.0486 166.67 4000 0.2967 0.5973 0.3964 0.2973
0.0381 187.5 4500 0.3066 0.6007 0.4060 0.3003
0.0392 208.33 5000 0.3054 0.6032 0.4055 0.2164
0.0268 229.17 5500 0.3052 0.6158 0.4081 0.3159
0.029 250.0 6000 0.2850 0.6292 0.3923 0.3108
0.0217 270.83 6500 0.2964 0.6141 0.3998 0.3130
0.0241 291.67 7000 0.3012 0.6216 0.4058 0.3197
0.038 312.5 7500 0.3051 0.6216 0.4093 0.2627
0.0374 333.33 8000 0.2914 0.6359 0.3997 0.3388
0.0194 354.17 8500 0.2975 0.6275 0.4037 0.3155
0.0189 375.0 9000 0.3037 0.625 0.4088 0.2911
0.0147 395.83 9500 0.2993 0.6242 0.4046 0.3417
0.0328 416.67 10000 0.3012 0.6242 0.4063 0.3210

Framework versions

  • Transformers 4.38.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.1
Downloads last month
15
Safetensors
Model size
287M params
Tensor type
F32
·
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for kavg/LiLT-RE-DE

Finetuned
(29)
this model
Finetunes
1 model