Thomas Lemberger commited on
Commit
d605b0e
1 Parent(s): 8a951fa
Files changed (1) hide show
  1. README.md +88 -0
README.md ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - english
4
+ thumbnail:
5
+ tags:
6
+ - token classification
7
+ license:
8
+ datasets:
9
+ - EMBO/sd-panels
10
+ metrics:
11
+ -
12
+ ---
13
+
14
+ # sd-ner
15
+
16
+ ## Model description
17
+
18
+ This model is a [RoBERTa base model](https://huggingface.co/roberta-base) that was further trained using a masked language modeling task on a compendium of english scientific textual examples from the life sciences using the [BioLang dataset](https://huggingface.co/datasets/EMBO/biolang) and fine-tuned for token classification on the SourceData [sd-panels](https://huggingface.co/datasets/EMBO/sd-panels) dataset to perform Named Entity Recognition of bioentities.
19
+
20
+
21
+ ## Intended uses & limitations
22
+
23
+ #### How to use
24
+
25
+ The intended use of this model is for Named Entity Recognition of biological entitie used in SourceData annotations (https://sourcedata.embo.org), including small molecules, gene products (genes and proteins), subcellular components, cell line and cell types, organ and tissues, species as well as experimental methods.
26
+
27
+ To have a quick check of the model:
28
+
29
+ ```python
30
+ from transformers import pipeline, RobertaTokenizerFast, RobertaForTokenClassification
31
+ example = """<s> F. Western blot of input and eluates of Upf1 domains purification in a Nmd4-HA strain. The band with the # might corresponds to a dimer of Upf1-CH, bands marked with a star correspond to residual signal with the anti-HA antibodies (Nmd4). Fragments in the eluate have a smaller size because the protein A part of the tag was removed by digestion with the TEV protease. G6PDH served as a loading control in the input samples </s>"""
32
+ tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base', max_len=512)
33
+ model = RobertaForTokenClassification.from_pretrained('EMBO/sd-ner')
34
+ ner = pipeline('ner', model, tokenizer=tokenizer)
35
+ res = ner(example)
36
+ for r in res:
37
+ print(r['word'], r['entity'])
38
+ ```
39
+
40
+ #### Limitations and bias
41
+
42
+ The model must be used with the `roberta-base` tokenizer.
43
+
44
+ ## Training data
45
+
46
+ The model was trained for token classification using the [EMBO/sd-panels dataset](https://huggingface.co/datasets/EMBO/biolang) wich includes manually annotated examples.
47
+
48
+ ## Training procedure
49
+
50
+ The training was run on a NVIDIA DGX Station with 4XTesla V100 GPUs.
51
+
52
+ Training code is available at https://github.com/source-data/soda-roberta
53
+
54
+ - Command: `python -m tokcl.train /data/json/sd_panels NER --num_train_epochs=3.5`
55
+ - Tokenizer vocab size: 50265
56
+ - Training data: EMBO/biolang MLM
57
+ - Training with 31410 examples.
58
+ - Evaluating on 8861 examples.
59
+ - Training on 15 features: O, I-SMALL_MOLECULE, B-SMALL_MOLECULE, I-GENEPROD, B-GENEPROD, I-SUBCELLULAR, B-SUBCELLULAR, I-CELL, B-CELL, I-TISSUE, B-TISSUE, I-ORGANISM, B-ORGANISM, I-EXP_ASSAY, B-EXP_ASSAY
60
+ - Epochs: 3.5
61
+ - `per_device_train_batch_size`: 32
62
+ - `per_device_eval_batch_size`: 32
63
+ - `learning_rate`: 0.0001
64
+ - `weight_decay`: 0.0
65
+ - `adam_beta1`: 0.9
66
+ - `adam_beta2`: 0.999
67
+ - `adam_epsilon`: 1e-08
68
+ - `max_grad_norm`: 1.0
69
+
70
+ ## Eval results
71
+
72
+ On test set with `sklearn.metrics`:
73
+
74
+ ```
75
+ precision recall f1-score support
76
+
77
+ CELL 0.77 0.81 0.79 3477
78
+ EXP_ASSAY 0.71 0.70 0.71 7049
79
+ GENEPROD 0.86 0.90 0.88 16140
80
+ ORGANISM 0.80 0.82 0.81 2759
81
+ SMALL_MOLECULE 0.78 0.82 0.80 4446
82
+ SUBCELLULAR 0.71 0.75 0.73 2125
83
+ TISSUE 0.70 0.75 0.73 1971
84
+
85
+ micro avg 0.79 0.82 0.81 37967
86
+ macro avg 0.76 0.79 0.78 37967
87
+ weighted avg 0.79 0.82 0.81 37967
88
+ ```