|
--- |
|
license: cc-by-nc-nd-4.0 |
|
widget: |
|
- text: "hemithyroidectomy is the removal of half of the <mask> gland." |
|
example_title: "Example 1" |
|
- text: "Li-Fraumeni <mask> is a hereditary tumor with autosomal dominant inheritance." |
|
example_title: "Example 2" |
|
- text: "The fascia in front of the pancreas was cut to the spleen direction to <mask> the splenic artery and vein." |
|
example_title: "Example 3" |
|
--- |
|
# SurgicBERTa |
|
SurgicBERTa is a language model based on RoBERTa-base (Liu et al., 2019) architecture. |
|
We adapted RoBERTa-base to different **surgical textbooks and academic papers** via continued pretraining. This amount to about 7 million words and 300k surgical sentences. |
|
We used the full text of the books and papers in training, not just abstracts. |
|
Specific details of the adaptive pretraining procedure and evaluation tasks can be found in the paper below cited. |
|
|
|
# Citation |
|
If using this model, please cite the following paper: |
|
<em> |
|
<br /> |
|
<br /> |
|
@article{bombieri_et_al_SurgicBERTa_2023, <br /> |
|
title = {Surgicberta: a pre-trained language model for procedural surgical language}, <br /> |
|
journal = {International Journal of Data Science and Analytics}, <br /> |
|
year = {2023}, <br /> |
|
doi = { https://doi.org/10.1007/s41060-023-00433-5 }, <br /> |
|
url = { https://link.springer.com/article/10.1007/s41060-023-00433-5 }, <br /> |
|
author = {Marco Bombieri and Marco Rospocher and Simone Paolo Ponzetto and Paolo Fiorini}, <br /> |
|
} |
|
</em> |
|
|
|
If using this model for Semantic Role Labeling, please cite also the following paper: |
|
<em> |
|
<br /> |
|
<br /> |
|
@article{bombieri_et_al_surgical_srl_2023, <br /> |
|
title = {Machine understanding surgical actions from intervention procedure textbooks}, <br /> |
|
journal = {Computers in Biology and Medicine}, <br /> |
|
volume = {152}, <br /> |
|
pages = {106415}, <br /> |
|
year = {2023}, <br /> |
|
issn = {0010-4825}, <br /> |
|
doi = {https://doi.org/10.1016/j.compbiomed.2022.106415}, <br /> |
|
url = {https://www.sciencedirect.com/science/article/pii/S0010482522011234}, <br /> |
|
author = {Marco Bombieri and Marco Rospocher and Simone Paolo Ponzetto and Paolo Fiorini}, <br /> |
|
keywords = {Semantic role labeling, Surgical data science, Procedural knowledge, Information extraction, Natural language processing} <br /> |
|
} |
|
</em> |
