RMU: bio forget corpora. cyber: cyber forget. both wikitext retain
https://arxiv.org/abs/2410.19278.
The code https://github.com/AMindToThink/wmdp.
-
AMindToThink/gemma-2-2b-it_RMU_s100_a300_layer3
Text Generation • Updated • 18 -
AMindToThink/gemma-2-2b-it_RMU_s100_a100_layer3
Text Generation • Updated • 19 -
AMindToThink/gemma-2-2b-it_RMU_s100_a500_layer3
Text Generation • Updated • 11 -
AMindToThink/gemma-2-2b-it_RMU_s100_a1200_layer3
Text Generation • Updated • 7