metadata

library_name: transformers
license: apache-2.0
base_model: answerdotai/ModernBERT-base
tags:
  - generated_from_trainer
metrics:
  - precision
  - recall
  - f1
  - accuracy
model-index:
  - name: edu-modernbert
    results: []

edu-modernbert

This model is a fine-tuned version of answerdotai/ModernBERT-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.2864
Precision: 0.5389
Recall: 0.3949
F1: 0.4305
Accuracy: 0.6820
Binary Precision: 0.7559
Binary Recall: 0.4496
Binary F1: 0.5638
Binary Accuracy: 0.9373

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 1024
eval_batch_size: 512
seed: 0
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy	Binary Precision	Binary Recall	Binary F1	Binary Accuracy
No log	0	0	2.4591	0.1116	0.1645	0.0484	0.1386	0.0	0.0	0.0	0.9098
0.3196	2.4331	1000	0.3097	0.5201	0.3691	0.3946	0.6587	0.7614	0.3580	0.4870	0.9320
0.3064	4.8662	2000	0.3067	0.5273	0.3882	0.4154	0.6599	0.7391	0.4375	0.5496	0.9353
0.3088	7.2993	3000	0.2951	0.5353	0.3833	0.4169	0.6744	0.7656	0.4007	0.5261	0.9349
0.2991	9.7324	4000	0.2975	0.5421	0.3921	0.4234	0.6699	0.7316	0.4643	0.5681	0.9363
0.2957	12.1655	5000	0.2920	0.5362	0.3859	0.4207	0.6813	0.7811	0.3953	0.5249	0.9355
0.2932	14.5985	6000	0.2881	0.5364	0.3946	0.4298	0.6824	0.7591	0.4351	0.5532	0.9366
0.2862	17.0316	7000	0.2876	0.5411	0.3850	0.4213	0.6829	0.7713	0.4104	0.5358	0.9359
0.2894	19.4647	8000	0.2864	0.5389	0.3949	0.4305	0.6820	0.7559	0.4496	0.5638	0.9373

Framework versions

Transformers 4.48.0.dev0
Pytorch 2.5.1+cu121
Datasets 3.2.0
Tokenizers 0.21.0