|
<!DOCTYPE html> |
|
<html lang="en"> |
|
<head> |
|
<meta charset="UTF-8"> |
|
<title>Title</title> |
|
</head> |
|
<body> |
|
This is a demo of <a href="https://arxiv.org/pdf/2209.11228.pdf">NamedMask: Distilling Segmenters from Complementary Foundation Models</a>.</br> |
|
The goal of this work is to segment and name regions of images without access to pixel-level labels during training. |
|
To tackle this task, we construct segmenters by distilling the complementary strengths of two foundation models. |
|
The first, CLIP (Radford et al. 2021), exhibits the ability to assign names to image content but lacks an accessible representation of object structure. |
|
The second, DINO (Caron et al. 2021), captures the spatial extent of objects but has no knowledge of object names. |
|
Our method, termed NamedMask, begins by using CLIP to construct category-specific archives of images. |
|
These images are pseudo-labelled with a category-agnostic salient object detector bootstrapped from DINO, then refined by category-specific segmenters using the CLIP archive labels. |
|
Thanks to the high quality of the refined masks, we show that a standard segmentation architecture trained on these archives with appropriate data augmentation achieves impressive semantic segmentation abilities for both single-object and multi-object images. |
|
As a result, our proposed NamedMask performs favourably against a range of prior work on five benchmarks including the VOC2012, COCO and large-scale ImageNet-S datasets. |
|
Code is publicly available at <a href="https://github.com/NoelShin/namedmask">our repo</a>. |
|
</body> |
|
</html> |
