links-ads/gaia-growseg

GRowSeg

Grapevine Rows Segmentation (GRowSeg)

The paper will be published soon.

NEW: check out our demo!

Table of Contents

1. Description and use cases
2. Model
3. Ipnut
4. Preprocessing
5. Output
6. Postprocessing
7. Dataset and training details
8. How to run GRowSeg
9. References
10. Contributors
11. License

Description and use cases

GRowSeg is a deep learning model for segmenting grapevine rows in UAV-acquired RGB images of vineyards. It takes an RGB orthoimage as input and predicts a binary segmentation mask, i.e. an image with '1' for rows and '0' for background.

Model

GRowSeg is a Segformer-b5 model. To allow comparison with previous state of the art (https://arxiv.org/pdf/2108.01200), the same experiments are performed. In particular, we repeat experiments from T1 to T4 of the reference paper with GRowSeg. Results are reported in the following table, in terms of F1 score.

Model	T1	T2	T3	T4
SegNet	0.73	0.85	0.85	0.76
UNet	0.75	0.82	0.91	0.75
ModSegNet	0.75	0.83	0.89	0.76
GRowSeg	0.78	0.85	0.91	0.78

Input

GRowSeg pipeline expects RGB input orthoimages in the uint8 format. The range of supported ground sampling distance (GSD) values is approximately [0.75, 10] cm/px.

Preprocessing

The input image, which has size HxWx3, is resized with the given scaling_factor, minimally padded compatibly with given patch_size and stride, scaled to [0, 1] and finally normalized with ImageNet mean and std. A moving window mechanism extract image tiles with size patch_size and overlapping with stride pixels, forming batches of batch_size image tiles. Each batch has therefore shape

int[batch_size, 3, patch_size, patch_size]

Output

Given a batch, the model outputs a pixel-wise confidence score map for each tile: each value represents the confidence of assigning that pixel to the 'vine' (1) class. The output batch has thus shape

int[batch_size, 1, patch_size, patch_size]

Postprocessing

Tiles are merged back together, averaging overlapping confidence scores (if stride != patch_size). The merged confidence score map is squeezed, unpadded and resized back to the original resolution HxW of the input orthoimage. A simple threshold at 0.5 is performed to convert the confidence score map to a binary segmentation mask. This mask is finally saved to the specified output path. If the input image is a georeferenced TIFF, the saved mask will be a georeferenced TIFF too.

Dataset and training details

The datasets used for training and testing GRowSeg can be found at:

• Group A orthoimages (request it to the owner of the repo)
• Group B orthoimages

How to run GRowSeg

• First, clone this repository:

git lfs install
git clone [email protected]:links-ads/vitigeoss-growseg

• Then, create a Python virtual environment and install dependencies:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Finally, to run GRowSeg on an input image:

python main.py "/path/to/input_image.tif" "/path/to/output_mask.tif"

The output path where to save the output mask must always be specified. The output filename must match the extension of the input filename. Supported image formats are .tif (preferred), .png, .jpg.

Several options can be specified:

• --patch_size: the resolution of the tiles extracted by the moving window (default: 512)
• --stride: the stride of the moving window (default: 256)
• --scaling_factor: scaling factor for resizing the image (default: 1.0)
• --rotate: perform inference also on 90°, 180°, 270°-rotated tiles, to enhance robustness at the cost of increased compute (default: False)
• --batch_size: batch size for the inference (default: 16)
• --verbose: tracks the inference with a progress bar (default: False)

Caveat:

• since GRowSeg was trained with patch_size = 512, it is suggested to leave it as default.
• the optimal GSD range for GrowSeg is [1, 1.5] cm/px. Therefore, you may want to rescale your image based on its gsd, by setting scaling_factor e.g. to GSD / 1.5
• GRowSeg automatically uses a GPU, if one is available on your PC. If this is the case, you may want to increase the batch_size to speed up inference

References

• GRowSeg presented in the Official GRowSeg repo

Contributors

• tommonopolinks (LINKS Foundation)
• FedericOldani (LINKS Foundation)

License

MIT License