Automated cell nuclei segmentation and classification

Models of the tumourkit library. The key idea behind these models is illustrated by the following image.

The objective is to detect and classify cells of different tissues. Different models trained with tissue from different organs and stainings are provided.

Lung (H&E)

Breast (HER2)

Consep: Colorectal (H&E)

Monusac: Miscelaneous (H&E)

Model description

The model is made by Hovernet as a backbone and a graph neural network on top to improve the classification step. Each backbone comes trained at two resolutions: 270x270 and 518x518. They also come in two version each, trained from scratch or fine-tuned from the consep checkpoint of Hovernet (FT). Then, for each Hovernet model, five graph neural networks are provided that can be used on top. Four graph convolutional neural networks trained with different sets of features and one graph attention network trained with all the features.

To use the models the tumourkit library comes with a simple demo that you can try. Beware, on CPU it takes nearly 10 minutes per 1024x1024 image.

Uses

Intended use

The lung models are built to estimate the percentage of tumoural cells in a given whole slide image (WSI). It is supposed to be used to accelerate histologist work and give priorities among huge amounts of WSIs to analyse.

The other three models are provided for research purposes only.

Misuse

By no means these models are supposed to substitute a medical expert, and they are not built for diagnosis. Usage in any critical situation is discouraged.

Citation

@article{PerezCano2024,
  author = {Jose Pérez-Cano and Irene Sansano Valero and David Anglada-Rotger and Oscar Pina and Philippe Salembier and Ferran Marques},
  title = {Combining graph neural networks and computer vision methods for cell nuclei classification in lung tissue},
  journal = {Heliyon},
  year = {2024},
  volume = {10},
  number = {7},
  doi = {10.1016/j.heliyon.2024.e28463},
}