CARIS: Context-Aware Referring Image Segmentation
This repository is for the ACM MM 2023 paper "CARIS: Context-Aware Referring Image Segmentation". Code is available here.
Pretrained CARIS Models
We have released pretrained CARIS models including:
- standard: 4 models trained on the training split of RefCOCO/RefCOCO+/RefCOCOg separately,
- combined: 1 model trained on the combination of the training splits of RefCOCO/RefCOCO+/RefCOCOg(umd split).
Performance (standard)
| Dataset | [email protected] | [email protected] | [email protected] | Overall IoU | Mean IoU |
|---|---|---|---|---|---|
| RefCOCO val | 87.92 | 80.99 | 39.54 | 75.13 | 77.41 |
| RefCOCO testA | 91.13 | 84.55 | 40.06 | 78.19 | 79.71 |
| RefCOCO testB | 83.75 | 84.82 | 40.47 | 71.75 | 74.51 |
| RefCOCO+ val | 79.97 | 73.54 | 36.15 | 66.63 | 70.66 |
| RefCOCO+ testA | 85.33 | 79.17 | 36.87 | 72.17 | 74.82 |
| RefCOCO+ testB | 71.81 | 63.51 | 33.48 | 58.54 | 63.97 |
| RefCOCOg val(U) | 77.39 | 68.57 | 32.62 | 64.32 | 68.61 |
| RefCOCOg test(U) | 78.11 | 69.06 | 32.27 | 65.82 | 69.02 |
| RefCOCOg val(G) | 76.72 | 68.72 | 34.03 | 63.84 | 67.80 |
Performance (combined)
| Dataset | [email protected] | [email protected] | [email protected] | Overall IoU | Mean IoU |
|---|---|---|---|---|---|
| RefCOCO val | 88.94 | 81.93 | 40.75 | 76.63 | 78.49 |
| RefCOCO testA | 91.50 | 84.80 | 40.14 | 79.40 | 80.21 |
| RefCOCO testB | 84.51 | 76.25 | 41.92 | 73.52 | 75.42 |
| RefCOCO+ val | 81.07 | 74.56 | 37.00 | 68.02 | 71.81 |
| RefCOCO+ testA | 86.10 | 80.00 | 37.98 | 73.69 | 75.75 |
| RefCOCO+ testB | 72.76 | 65.13 | 35.51 | 60.41 | 65.15 |
| RefCOCOg val(U) | 80.84 | 71.73 | 35.07 | 67.95 | 71.53 |
| RefCOCOg test(U) | 81.91 | 73.22 | 34.95 | 69.75 | 72.18 |