pages/Backbone/Backbone.md

Many cutting-edge computer vision models consist of multiple stages:
➰ backbone extracts the features,
➰ neck refines the features,
➰ head makes the detection for the task.
Implementing this is cumbersome, so 🤗 transformers has an API for this: Backbone!

Let's see an example of such model. Assuming we would like to initialize a multi-stage instance segmentation model with ResNet backbone and MaskFormer neck and a head, you can use the backbone API like following (left comments for clarity) 👇

One can also use a backbone just to get features from any stage. You can initialize any backbone with AutoBackbone class. See below how to initialize a backbone and getting the feature maps at any stage 👇

Backbone API also supports any timm backbone of your choice! Check out a variation of timm backbones here.

Leaving some links 🔗:
📖 I've created a notebook for you to play with it
📒 Backbone API docs 📓 AutoBackbone docs 💜
(all written with love by me!)

Orignial tweet (January 23, 2024)