tomaarsen/span-marker-bert-base-uncased-keyphrase-inspec

SpanMarker with bert-base-uncased on Inspec

This is a SpanMarker model trained on the Inspec dataset that can be used for Named Entity Recognition. This SpanMarker model uses bert-base-uncased as the underlying encoder. See train.py for the training script.

Model Details

Model Description

• Model Type: SpanMarker
• Encoder: bert-base-uncased
• Maximum Sequence Length: 256 tokens
• Maximum Entity Length: 8 words
• Training Dataset: Inspec
• Language: en
• License: apache-2.0

Model Sources

• Repository: SpanMarker on GitHub
• Thesis: SpanMarker For Named Entity Recognition

Model Labels

Label	Examples
KEY	"Content Atomism", "philosophy of mind", "IBS"

Evaluation

Metrics

Label	Precision	Recall	F1
all	0.5666	0.6230	0.5935
KEY	0.5666	0.6230	0.5935

Uses

Direct Use

from span_marker import SpanMarkerModel

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-uncased-keyphrase-inspec")
# Run inference
entities = model.predict("Adaptive filtering for noise reduction in hue saturation intensity color space Even though the hue saturation intensity -LRB- HSI -RRB- color model has been widely used in color image processing and analysis, the conversion formulas from the RGB color model to HSI are nonlinear and complicated in comparison with the conversion formulas of other color models. When an RGB image is degraded by random Gaussian noise, this nonlinearity leads to a nonuniform noise distribution in HSI, making accurate image analysis more difficult. We have analyzed the noise characteristics of the HSI color model and developed an adaptive spatial filtering method to reduce the magnitude of noise and the nonuniformity of noise variance in the HSI color space. With this adaptive filtering method, the filter kernel for each pixel is dynamically adjusted, depending on the values of intensity and saturation. In our experiments we have filtered the saturation and hue components and generated edge maps from color gradients. We have found that by using the adaptive filtering method, the minimum error rate in edge detection improves by approximately 15%")

Downstream Use

You can finetune this model on your own dataset.

Click to expand

from span_marker import SpanMarkerModel, Trainer

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-uncased-keyphrase-inspec")

# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003

# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
    model=model,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("tomaarsen/span-marker-bert-base-uncased-keyphrase-inspec-finetuned")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Sentence length	15	138.5327	557
Entities per sentence	0	8.2507	41

Training Hyperparameters

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3

Environmental Impact

Carbon emissions were measured using CodeCarbon.

• Carbon Emitted: 0.021 kg of CO2
• Hours Used: 0.137 hours

Training Hardware

• On Cloud: No
• GPU Model: 1 x NVIDIA GeForce RTX 3090
• CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
• RAM Size: 31.78 GB

Framework Versions

• Python: 3.9.16
• SpanMarker: 1.3.1.dev
• Transformers : 4.29.2
• PyTorch: 2.0.1+cu118
• Datasets: 2.14.3
• Tokenizers: 0.13.2