Coreference resolution model for the Ukrainian language
The coreference resolution model for the Ukrainian language was trained on the silver Ukrainian coreference dataset using the F-Coref library. The model was trained on top of the XML-Roberta-base model.
Model Details
Model Description
- Developed by: Artem Kramov, Andrii Kursin ([email protected]).
- Languages: Ukrainian
- Finetuned from model: XML-Roberta-base
Model Sources
- Repository: https://github.com/artemkramov/fastcoref-ua/blob/main/README.md
- Demo: Google Colab, Streamlit
Out-of-Scope Use
According to the metrics retrieved from the evaluation dataset, the model is more precision-oriented. Also, there is a high level of granularity of mentions. E.g., the mention "Головний виконавчий директор Андрій Сидоренко" can be divided into the following coreferent groups: ["Головний виконавчий директор Андрій Сидоренко", "Головний виконавчий директор", "Андрій Сидоренко"]. Such a feature can also be used to extract some positions, roles, or other features of entities in the text.
How to Get Started with the Model
Use the code below to get started with the model.
from fastcoref import FCoref
import spacy
nlp = spacy.load('uk_core_news_md')
model_path = "artemkramov/coref-ua"
model = FCoref(model_name_or_path=model_path, device='cuda:0', nlp=nlp)
preds = model.predict(
texts=["""Мій друг дав мені свою машину та ключі до неї; крім того, він дав мені його книгу. Я з радістю її читаю."""]
)
preds[0].get_clusters(as_strings=False)
> [[(0, 3), (13, 17), (66, 70), (83, 84)],
[(0, 8), (18, 22), (58, 61), (71, 75)],
[(18, 29), (42, 45)],
[(71, 81), (95, 97)]]
preds[0].get_clusters()
> [['Мій', 'мені', 'мені', 'Я'], ['Мій друг', 'свою', 'він', 'його'], ['свою машину', 'неї'], ['його книгу', 'її']]
preds[0].get_logit(
span_i=(13, 17), span_j=(42, 45)
)
> -6.867196
Training Details
Training Data
The model was trained on the silver coreference resolution dataset: https://huggingface.co/datasets/artemkramov/coreference-dataset-ua.
Evaluation
Metrics
Two types of metrics were considered: mention-based and the coreference resolution metrics themselves.
Mention-based metrics:
- mention precision
- mention recall
- mention F1
Coreference resolution metrics were calculated as the average values across the following metrics: MUC, BCubed, CEAFE:
- coreference precision
- coreference recall
- coreference F1
Results
The metrics for the validation dataset:
Metric | Value |
---|---|
Mention precision | 0.850 |
Mention recall | 0.798 |
Mention F1 | 0.824 |
Coreference precision | 0.758 |
Coreference recall | 0.706 |
Coreference F1 | 0.731 |
Model Card Authors
Artem Kramov (https://www.linkedin.com/in/artem-kramov-0b3731100/), Andrii Kursin ([email protected])
- Downloads last month
- 11
Dataset used to train artemkramov/coref-ua
Space using artemkramov/coref-ua 1
Evaluation results
- Mean F1 measure of MUC, BCubed, and CEAFE on Silver Ukrainian Coreference Resolution Datasetself-reported0.731