|
--- |
|
language: |
|
- en |
|
pipeline_tag: image-segmentation |
|
--- |
|
# CARIS: Context-Aware Referring Image Segmentation |
|
This repository is for the ACM MM 2023 paper "CARIS: Context-Aware Referring Image Segmentation". Code is available [here](https://github.com/lsa1997/CARIS). |
|
|
|
## Pretrained CARIS Models |
|
We have released pretrained CARIS models including: |
|
- standard: 4 models trained on the training split of RefCOCO/RefCOCO+/RefCOCOg separately, |
|
- combined: 1 model trained on the combination of the training splits of RefCOCO/RefCOCO+/RefCOCOg(umd split). |
|
|
|
## Performance (standard) |
|
| Dataset | P\@0.5 | P\@0.7 |P\@0.9 | Overall IoU | Mean IoU | |
|
|:---------------:|:-----:|:-----:|:-----:|:-----------:|:--------:| |
|
| RefCOCO val | 87.92 | 80.99 | 39.54 | 75.13 | 77.41 | |
|
| RefCOCO testA | 91.13 | 84.55 | 40.06 | 78.19 | 79.71 | |
|
| RefCOCO testB | 83.75 | 84.82 | 40.47 | 71.75 | 74.51 | |
|
| RefCOCO+ val | 79.97 | 73.54 | 36.15 | 66.63 | 70.66 | |
|
| RefCOCO+ testA | 85.33 | 79.17 | 36.87 | 72.17 | 74.82 | |
|
| RefCOCO+ testB | 71.81 | 63.51 | 33.48 | 58.54 | 63.97 | |
|
| RefCOCOg val(U) | 77.39 | 68.57 | 32.62 | 64.32 | 68.61 | |
|
| RefCOCOg test(U)| 78.11 | 69.06 | 32.27 | 65.82 | 69.02 | |
|
| RefCOCOg val(G) | 76.72 | 68.72 | 34.03 | 63.84 | 67.80 | |
|
|
|
## Performance (combined) |
|
| Dataset | P\@0.5 | P\@0.7 |P\@0.9 | Overall IoU | Mean IoU | |
|
|:---------------:|:-----:|:-----:|:-----:|:-----------:|:--------:| |
|
| RefCOCO val | 88.94 | 81.93 | 40.75 | 76.63 | 78.49 | |
|
| RefCOCO testA | 91.50 | 84.80 | 40.14 | 79.40 | 80.21 | |
|
| RefCOCO testB | 84.51 | 76.25 | 41.92 | 73.52 | 75.42 | |
|
| RefCOCO+ val | 81.07 | 74.56 | 37.00 | 68.02 | 71.81 | |
|
| RefCOCO+ testA | 86.10 | 80.00 | 37.98 | 73.69 | 75.75 | |
|
| RefCOCO+ testB | 72.76 | 65.13 | 35.51 | 60.41 | 65.15 | |
|
| RefCOCOg val(U) | 80.84 | 71.73 | 35.07 | 67.95 | 71.53 | |
|
| RefCOCOg test(U)| 81.91 | 73.22 | 34.95 | 69.75 | 72.18 | |
