---
language:
- en
- es
- fr
- nl
- de
- fi
- ru
- tr
- ko
- zh
- ja
- th
- pt
- it
- sv
- hu
- pl
- et
- hr
- uk
- el
- da
- he
tags:
- biomedical
- bionlp
- entity linking
- embedding
- bert
---

A multilingual BERGAMOT model with pre-trained on UMLS (version 2020AB) using a Graph Attention Network (GAT) encoder.

The model is described in paper "Biomedical Entity Representation with Graph-Augmented Multi-Objective Transformer " which is accepted to NAACL 2024!

For pretraining code see our github: [https://github.com/Andoree/BERGAMOT](https://github.com/Andoree/BERGAMOT).

Please do not forget to cite our paper when bib will be available.