Update README.md
Browse files
README.md
CHANGED
@@ -3,9 +3,9 @@ license: apache-2.0
|
|
3 |
---
|
4 |
# InterProt ESM2 SAE Models
|
5 |
|
6 |
-
A set of SAE models trained on [ESM2-650](https://huggingface.co/facebook/esm2_t33_650M_UR50D) activations using protein sequences from [UniProt](https://www.uniprot.org/).
|
7 |
|
8 |
-
Check out [https://interprot.com](https://interprot.com) for an interactive visualizer of the 4096-dimensional SAE.
|
9 |
|
10 |
## Installation
|
11 |
|
|
|
3 |
---
|
4 |
# InterProt ESM2 SAE Models
|
5 |
|
6 |
+
A set of SAE models trained on [ESM2-650](https://huggingface.co/facebook/esm2_t33_650M_UR50D) activations using 1M protein sequences from [UniProt](https://www.uniprot.org/). The SAE implementation mostly followed [Gao et al.](https://arxiv.org/abs/2406.04093) with Top-K activation function, though with much fewer latent dimensions.
|
7 |
|
8 |
+
Check out [https://interprot.com](https://interprot.com) for an interactive visualizer of the 4096-dimensional SAE on ESM layer 24.
|
9 |
|
10 |
## Installation
|
11 |
|