Multithreaded crawler with priority queue based on reliability of link sources

Ensure tokenizer is on GPU

Diagnostics

Diagnostics

Diagnostics

Diagnostics

Diagnostics

Ensure attention mask is assigned to GPU

Moved device assignement to beginning of function

Corrected typo

Ensure consistency of device assignment when training

Ensure consistency of device assignment when training

Fixed import

Fixed import

Factorized the weight matrix in the GlobalAttentionPoolingHead, thus reducing the number of parameters in this layer by a factor of 48

Clear cuda cache

Reduced batch size

Google Colab imports

Training on Colab. Half a crawler

Download training data from S3

Removed unnecessary import

Removed train_base_models task. Not needed with RoBERTa models

Added gradio to requirements.txt

Configured gradio app for training

Pin allennlp and torch to solve package conflics (#2)

Added HuggingFace Space configuration YAML to README.md

Added HuggingFace Space configuration YAML to README.md

Created HuggingFace Space

Removed diagnostic

Ensure consitency value is 32 bit

Removed diagnostics

Correct dimension of consistency cosine

Wrap CrossEntropyLoss in callable that makes it appplicable to sequences

Using torch.nn.CosineSimilarity to simplify code

Use builtin sum rather than torch.sum for a generator

Removed unnecessary parameters

Attention mask in decoder

Set use_cache argument

Fix Einstein summation notation

Use keepdim option when normalising vectors

Make EPSILON a tensor

torch.maximum, not torch.max

Unsqueeze attention mask

Unpack BatchEncoding

Create BatchEncoding in pad

Use BatchEncoding for training, not batch_encoding

Use BatchEncoding for training

Removed unnecessary on_epoch_end

Learning rate scheduler

input_embeddings not needed