Are we able to use pre-trained model to classify cells without the fine-tune layer?

#83
by DYXDAVE - opened

Hi
I'm new to transformer and I'm curious that can we get the output from the pre-trained model without adding the fine-tune layer? If we could do so, how should we set up our trainer/training_args?
Thanks

Thank you for your interest in Geneformer. You could extract cell embeddings from the frozen pretrained model and pass that to any classifier. However, the pretrained model was trained with a masked learning objective so it would be best fine-tuned towards the cell classification objective you are interested in. You can choose to freeze any number of layers (0, some, or all) during the fine-tuning. The last layer added during fine-tuning is for the classification itself.

ctheodoris changed discussion status to closed

Sign up or log in to comment