File size: 2,056 Bytes
6d4618c f569a3d 6d4618c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
---
language:
- en
pipeline_tag: image-segmentation
---
# CARIS: Context-Aware Referring Image Segmentation
This repository is for the ACM MM 2023 paper "CARIS: Context-Aware Referring Image Segmentation". Code is available [here](https://github.com/lsa1997/CARIS).
## Pretrained CARIS Models
We have released pretrained CARIS models including:
- standard: 4 models trained on the training split of RefCOCO/RefCOCO+/RefCOCOg separately,
- combined: 1 model trained on the combination of the training splits of RefCOCO/RefCOCO+/RefCOCOg(umd split).
## Performance (standard)
| Dataset | P\@0.5 | P\@0.7 |P\@0.9 | Overall IoU | Mean IoU |
|:---------------:|:-----:|:-----:|:-----:|:-----------:|:--------:|
| RefCOCO val | 87.92 | 80.99 | 39.54 | 75.13 | 77.41 |
| RefCOCO testA | 91.13 | 84.55 | 40.06 | 78.19 | 79.71 |
| RefCOCO testB | 83.75 | 84.82 | 40.47 | 71.75 | 74.51 |
| RefCOCO+ val | 79.97 | 73.54 | 36.15 | 66.63 | 70.66 |
| RefCOCO+ testA | 85.33 | 79.17 | 36.87 | 72.17 | 74.82 |
| RefCOCO+ testB | 71.81 | 63.51 | 33.48 | 58.54 | 63.97 |
| RefCOCOg val(U) | 77.39 | 68.57 | 32.62 | 64.32 | 68.61 |
| RefCOCOg test(U)| 78.11 | 69.06 | 32.27 | 65.82 | 69.02 |
| RefCOCOg val(G) | 76.72 | 68.72 | 34.03 | 63.84 | 67.80 |
## Performance (combined)
| Dataset | P\@0.5 | P\@0.7 |P\@0.9 | Overall IoU | Mean IoU |
|:---------------:|:-----:|:-----:|:-----:|:-----------:|:--------:|
| RefCOCO val | 88.94 | 81.93 | 40.75 | 76.63 | 78.49 |
| RefCOCO testA | 91.50 | 84.80 | 40.14 | 79.40 | 80.21 |
| RefCOCO testB | 84.51 | 76.25 | 41.92 | 73.52 | 75.42 |
| RefCOCO+ val | 81.07 | 74.56 | 37.00 | 68.02 | 71.81 |
| RefCOCO+ testA | 86.10 | 80.00 | 37.98 | 73.69 | 75.75 |
| RefCOCO+ testB | 72.76 | 65.13 | 35.51 | 60.41 | 65.15 |
| RefCOCOg val(U) | 80.84 | 71.73 | 35.07 | 67.95 | 71.53 |
| RefCOCOg test(U)| 81.91 | 73.22 | 34.95 | 69.75 | 72.18 | |