This is a converted version of Instadeep's TunBERT from nemo to safetensors.

Make sure to read the original model licence

architectural changes

original model head

image/png

this model head

image/png

Note

this is a WIP and any contributions are welcome

how to load the model

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("not-lain/TunBERT")
model = AutoModelForSequenceClassification.from_pretrained("not-lain/TunBERT",trust_remote_code=True)

how to use the model

text = "[insert text here]"
inputs = tokenizer(text,return_tensors='pt')
output = model(**inputs)

or you can use the pipeline :

from transformers import pipeline

pipe = pipeline(model="not-lain/TunBERT",tokenizer = "not-lain/TunBERT",trust_remote_code=True)
pipe("text")

IMPORTANT :

  • Make sure to enable trust_remote_code=True
Downloads last month
451
Safetensors
Model size
110M params
Tensor type
F32
·
Inference Examples
Inference API (serverless) does not yet support model repos that contain custom code.

Dataset used to train tunis-ai/TunBERT