google
/

gemma-scope-9b-pt-res

Model card Files Files and versions Community

gemma-scope-9b-pt-res / layer_18 /width_16k /average_l0_140 /hparams.json

Tom Lieberum

9B: Add sparsity lambdas for residual stream and clean up the feature splitting suite so that we only have SAEs with learning rate 7e-5

745af8a 4 days ago

28 Bytes

{"sparsity_lambda": 0.00025}