Are we able to use pre-trained model to classify cells without the fine-tune layer?
#83
by
DYXDAVE
- opened
Hi
I'm new to transformer and I'm curious that can we get the output from the pre-trained model without adding the fine-tune layer? If we could do so, how should we set up our trainer/training_args?
Thanks
Thank you for your interest in Geneformer. You could extract cell embeddings from the frozen pretrained model and pass that to any classifier. However, the pretrained model was trained with a masked learning objective so it would be best fine-tuned towards the cell classification objective you are interested in. You can choose to freeze any number of layers (0, some, or all) during the fine-tuning. The last layer added during fine-tuning is for the classification itself.
ctheodoris
changed discussion status to
closed