i-be-snek/distilbert-base-uncased-finetuned-ner-exp_A

This model is a fine-tuned version of distilbert-base-uncased on the English subset of all named entities in Babelscape/multinerd dataset. It achieves the following results on the validation set:

• Train Loss: 0.0163
• Validation Loss: 0.1024
• Train Precision: 0.8763
• Train Recall: 0.8862
• Train F1: 0.8812
• Train Accuracy: 0.9750
• Epoch: 2

Model description

distilbert-base-uncased-finetuned-ner-exp_A is a Named Entity Recognition model finetuned on distilbert-base-uncased. This model is uncased, so it makes no distinction between "sarah" and "Sarah".

Training and evaluation data

This model has been evaluated on the English subset of the test set of Babelscape/multinerd

Evaluation results

metric	value
precision	0.905358
recall	0.930318
f1	0.917668
accuracy	0.986355

metric/tag	ANIM	BIO	CEL	DIS	EVE	FOOD	INST	LOC	MEDIA	MYTH	ORG	PER	PLANT	TIME	VEHI
precision	0.667262	0.666667	0.508197	0.662324	0.896277	0.637809	0.642857	0.964137	0.931915	0.638889	0.941176	0.99033	0.558043	0.756579	0.735294
recall	0.698878	0.75	0.756098	0.803689	0.957386	0.637809	0.75	0.963656	0.956332	0.71875	0.962224	0.992023	0.752796	0.795848	0.78125
f1	0.682704	0.705882	0.607843	0.72619	0.925824	0.637809	0.692308	0.963897	0.943966	0.676471	0.951584	0.991176	0.640952	0.775717	0.757576
number	3208	16	82	1518	704	1132	24	24048	916	64	6618	10530	1788	578	64

Training procedure

All scripts for training can be found in this GitHub repository. The model had early stopped watching its val_loss.

Training hyperparameters

The following hyperparameters were used during training:

• optimizer:

    {
        "name": "AdamWeightDecay",
        "learning_rate": 2e-05,
        "decay": 0.0,
        "beta_1": 0.9,
        "beta_2": 0.999,
        "epsilon": 1e-07,
        "amsgrad": False,
        "weight_decay_rate": 0.0,
    }
  

• training_precision: float32

Training results

Train Loss	Validation Loss	Train Precision	Train Recall	Train F1	Train Accuracy	Epoch
0.0709	0.0710	0.8563	0.8875	0.8716	0.9735	0
0.0295	0.0851	0.8743	0.8835	0.8789	0.9748	1
0.0163	0.1024	0.8763	0.8862	0.8812	0.9750	2

Epoch 0

Named Entity	precision	recall	f1
ANIM	0.699150	0.620124	0.657270
BIO	0.480000	0.782609	0.595041
CEL	0.815385	0.876033	0.844622
DIS	0.628939	0.806709	0.706818
EVE	0.898876	0.924855	0.911681
FOOD	0.624774	0.602266	0.613314
INST	0.467391	0.741379	0.573333
LOC	0.967354	0.969634	0.968493
MEDIA	0.911227	0.939856	0.925320
MYTH	0.941860	0.771429	0.848168
ORG	0.924471	0.937629	0.931003
PER	0.988699	0.990918	0.989807
PLANT	0.622521	0.781333	0.692944
TIME	0.743902	0.738499	0.741191
VEHI	0.785714	0.791367	0.788530

Epoch 1

Named Entity	precision	recall	f1
ANIM	0.701040	0.747340	0.723450
BIO	0.422222	0.826087	0.558824
CEL	0.729167	0.867769	0.792453
DIS	0.731099	0.749794	0.740328
EVE	0.864865	0.924855	0.893855
FOOD	0.652865	0.572632	0.610122
INST	0.871795	0.586207	0.701031
LOC	0.968255	0.966143	0.967198
MEDIA	0.946346	0.918312	0.932118
MYTH	0.914894	0.819048	0.864322
ORG	0.906064	0.943582	0.924442
PER	0.990389	0.988367	0.989377
PLANT	0.625889	0.743556	0.679667
TIME	0.755981	0.765133	0.760529
VEHI	0.737500	0.848921	0.789298

Epoch 2

Named Entity	precision	recall	f1
ANIM	0.730443	0.687057	0.708086
BIO	0.330882	0.978261	0.494505
CEL	0.798561	0.917355	0.853846
DIS	0.738108	0.750894	0.744446
EVE	0.904899	0.907514	0.906205
FOOD	0.628664	0.623184	0.625912
INST	0.533333	0.551724	0.542373
LOC	0.967915	0.973997	0.970946
MEDIA	0.949627	0.913824	0.931382
MYTH	0.910000	0.866667	0.887805
ORG	0.924920	0.934136	0.929505
PER	0.989506	0.991020	0.990263
PLANT	0.637648	0.742222	0.685972
TIME	0.766355	0.794189	0.780024
VEHI	0.818182	0.647482	0.722892

Framework versions

• Transformers 4.35.2
• TensorFlow 2.14.0
• Datasets 2.15.0
• Tokenizers 0.15.0