jayant-yadav
commited on
Commit
·
9500fd3
1
Parent(s):
59a6d3c
updated code to run pipeline
Browse files
README.md
CHANGED
@@ -95,8 +95,8 @@ Use the code below to get started with the model:
|
|
95 |
from transformers import AutoTokenizer, AutoModelForTokenClassification
|
96 |
from transformers import pipeline
|
97 |
|
98 |
-
tokenizer = AutoTokenizer.from_pretrained("
|
99 |
-
model = AutoModelForTokenClassification.from_pretrained("
|
100 |
|
101 |
nlp = pipeline("ner", model=model, tokenizer=tokenizer)
|
102 |
example = "My name is Wolfgang and I live in Berlin"
|
@@ -116,7 +116,7 @@ print(ner_results)
|
|
116 |
|
117 |
#### Preprocessing [optional]
|
118 |
|
119 |
-
English dataset was filterd out :
|
120 |
|
121 |
|
122 |
#### Training Hyperparameters
|
@@ -146,8 +146,8 @@ Tested on Full test split of MultiNERD dataset.
|
|
146 |
|
147 |
|
148 |
#### Metrics
|
149 |
-
Model versions and checkpoint were evaluated using F1, Precision and Recall.
|
150 |
-
For this `seqeval` metric was used:
|
151 |
|
152 |
### Results
|
153 |
|
@@ -179,7 +179,7 @@ Follows the same as RoBERTa-BASE
|
|
179 |
|
180 |
#### Hardware
|
181 |
|
182 |
-
Kaggle - GPU T4x2
|
183 |
Google Colab - GPU T4x1
|
184 |
|
185 |
#### Software
|
|
|
95 |
from transformers import AutoTokenizer, AutoModelForTokenClassification
|
96 |
from transformers import pipeline
|
97 |
|
98 |
+
tokenizer = AutoTokenizer.from_pretrained("jayant-yadav/roberta-base-multinerd")
|
99 |
+
model = AutoModelForTokenClassification.from_pretrained("jayant-yadav/roberta-base-multinerd")
|
100 |
|
101 |
nlp = pipeline("ner", model=model, tokenizer=tokenizer)
|
102 |
example = "My name is Wolfgang and I live in Berlin"
|
|
|
116 |
|
117 |
#### Preprocessing [optional]
|
118 |
|
119 |
+
English dataset was filterd out : ```train_dataset = train_dataset.filter(lambda x: x['lang'] == 'en')```
|
120 |
|
121 |
|
122 |
#### Training Hyperparameters
|
|
|
146 |
|
147 |
|
148 |
#### Metrics
|
149 |
+
Model versions and checkpoint were evaluated using F1, Precision and Recall.
|
150 |
+
For this `seqeval` metric was used: ```metric = load_metric("seqeval")```.
|
151 |
|
152 |
### Results
|
153 |
|
|
|
179 |
|
180 |
#### Hardware
|
181 |
|
182 |
+
Kaggle - GPU T4x2
|
183 |
Google Colab - GPU T4x1
|
184 |
|
185 |
#### Software
|