Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddings
Paper
•
2305.13571
•
Published
•
2
Papers - Text - Encoders - Bert
-
-
BERTs are Generative In-Context Learners
Paper • 2406.04823 • Published • 1 -
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
Paper • 2412.13663 • Published • 103 -
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Paper • 2006.11477 • Published • 5