noanabeshima commited on
Commit
32b74e7
·
verified ·
1 Parent(s): b431aec

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -3
README.md CHANGED
@@ -28,6 +28,4 @@ feat_acts = load_feat_acts(f"mlp_map_test/M2_S-2_R1_P0/{300}.pt").to_dense()
28
  ```
29
 
30
  The activations are from 13% of the train split of https://huggingface.co/datasets/noanabeshima/TinyModelTokIds.
31
- I gather all the activations per feature and then take the smallest activations prefix so that each feature has at least 3K documents on which it activates and 12K token-activations.
32
-
33
- I think it takes something like 6 hours to do 9K features on a single 4090.
 
28
  ```
29
 
30
  The activations are from 13% of the train split of https://huggingface.co/datasets/noanabeshima/TinyModelTokIds.
31
+ I gather all the activations per feature and then take the smallest activations prefix so that each feature has at least 3K documents on which it activates and 12K token-activations.