File size: 2,744 Bytes
3efb6b1 5826d13 a7bc81a 5826d13 3efb6b1 d190c34 249546d d190c34 249546d d190c34 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
---
license: apache-2.0
language:
- pt
widget:
- text: "A pressão arterial está [MASK], indicando possível hipertensão."
- text: "O paciente recebeu [MASK] do hospital."
- text: "O médico receitou a medicação para controlar a [MASK]."
- text: "O paciente apresenta batimentos cardíacos irregulares, sugerindo [MASK]."
---
# CardioBERTpt - Portuguese Transformer-based Models for Clinical Language Representation in Cardiology
This model card describes CardioBERTpt, a clinical model trained on the cardiology domain for NER tasks in Portuguese. This model is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on a cardiology text dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4495
- Accuracy: 0.8864
## How to use the model
Load the model via the transformers library:
```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("pucpr-br/cardiobertpt")
model = AutoModel.from_pretrained("pucpr-br/cardiobertpt")
```
## Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15.0
## Framework versions
- Transformers 4.17.0.dev0
- Pytorch 1.8.0
- Datasets 1.18.3
- Tokenizers 0.11.0
## More Information
Refer to the original paper, [CardioBERTpt - Portuguese Transformer-based Models for Clinical Language Representation in Cardiology](https://ieeexplore.ieee.org/document/10178779/) for additional details and performance on Portuguese NER tasks.
## Acknowledgements
This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001, and by Foxconn Brazil and Zerbini Foundation as part of the research project Machine Learning in Cardiovascular Medicine.
## Citation
```
@INPROCEEDINGS{10178779,
author={Schneider, Elisa Terumi Rubel and Gumiel, Yohan Bonescki and de Souza, João Vitor Andrioli and Mie Mukai, Lilian and Emanuel Silva e Oliveira, Lucas and de Sa Rebelo, Marina and Antonio Gutierrez, Marco and Eduardo Krieger, Jose and Teodoro, Douglas and Moro, Claudia and Paraiso, Emerson Cabrera},
booktitle={2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS)},
title={CardioBERTpt: Transformer-based Models for Cardiology Language Representation in Portuguese},
year={2023},
volume={},
number={},
pages={378-381},
doi={10.1109/CBMS58004.2023.00247}}
}
```
## Questions?
Post a Github issue on the [CardioBERTpt repo](https://github.com/HAILab-PUCPR/CardioBERTpt). |