Removed unnecessary parameters

Attention mask in decoder

Set use_cache argument

input_embeddings not needed

Removed unnecessary parameter

get_input_embeddings() directly from base model

There's a simpler way of doing this, I hope

Might be simpler to inherit from RobertaModel rather than PreTrainedModel

Removed a base model that was causing a loop in model initialisation

Removed line that would have failed

Fixed import

Further changes for compatibility with HuggingFace Pytorch implementation

PyTorch implementation of HugggingFace PreTrainedModel class does not allow direct setting of base_model. Rejig constructors accordingly

Corrected inheritance

Converted QaracDecoderModel to use PyTorch

The other layer returned a tuple as well

Low level RoBERTa layers don't necessarily return what I expect them to

Fixed typo

Needed more arguments

Arguments to Concatenate layer should be in a list

Fixed arguments to decoder head

Attention masks, generation, and testing script

Making sure RoBERTa layers have all required arguments

Add an extra dimension to the input vector

Head needed to know input embeddings

Fix typo

Go to correct submodule

Fix typo

Fix typo

Find the layers I need

Find the layers I need

Use base model config in superclass __init__

Fix inheritance

Fix typos

Training script

More work on models

Encoder, Decoder and Trainer models (assuming RoBERTa base models)