How can we access the logits from this model output?
I want to access the mode = 'likelihood' from the paper equivalent from the Huggingface model
I think the model here just has the Abrep part that generates embeddings. How can we access the Abhead or upload our own version?
I used AbLang from original paper only for generating embeddings. I never looked for Abhead. I thing in your case you need to use original model
Hi, how did you upload the model weights from the original model in the paper to the HuggingFace hub? I want to add the model with AbHead (The likelihood part - aka decoder) and the AbRep (Embedding part aka encoder)
I looked at arcitecture and it was roberta but names of layers were different. And aslo original model had custom embedding layer and I used BERT tokenizer because in original paper they also used something similar and not roberta tokenizer.
This hf model only has encoder part to extract embeddings. I did not port AbHead.
Got it. Thank you so much this is super helpful!
I see that the AbHead is just a linear layer (768, 768) + normalization layer + another linear layer (768, 24) on top of the embedding (768, ) layer. Is it possible to add these layers on top of the repo you have created? How can we specifically extract these weights?