Spaces:
Paused
Paused
x-lai
commited on
Commit
·
524bb72
1
Parent(s):
11d7ed8
Update README.md
Browse filesFormer-commit-id: 5ce62aad828feabffe9deb2e4c34798b49038672
README.md
CHANGED
@@ -4,11 +4,57 @@
|
|
4 |
|
5 |
<font size=10><div align='center' > <a href=https://arxiv.org/pdf/2308.00692.pdf>**Paper**</a> | <a href="https://huggingface.co/xinlai">**Models**</a> | [**Inference**](#inference) | [**Dataset**](#dataset) | <a href="http://103.170.5.190:7860/">**Online Demo**</a></div></font>
|
6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
7 |
|
8 |
<p align="center"> <img src="imgs/fig_overview.jpg" width="100%"> </p>
|
9 |
|
10 |
-
<p align="center"> <img src="imgs/teaser.jpg" width="100%"> </p>
|
11 |
-
|
12 |
## News
|
13 |
- [x] [2023.8.4] [Online Demo](http://103.170.5.190:7860/) is released!
|
14 |
- [x] [2023.8.4] [*ReasonSeg* Dataset](https://drive.google.com/drive/folders/125mewyg5Ao6tZ3ZdJ-1-E3n04LGVELqy?usp=sharing) and the [LISA-13B-llama2-v0-explainatory](https://huggingface.co/xinlai/LISA-13B-llama2-v0-explainatory) model are released!
|
|
|
4 |
|
5 |
<font size=10><div align='center' > <a href=https://arxiv.org/pdf/2308.00692.pdf>**Paper**</a> | <a href="https://huggingface.co/xinlai">**Models**</a> | [**Inference**](#inference) | [**Dataset**](#dataset) | <a href="http://103.170.5.190:7860/">**Online Demo**</a></div></font>
|
6 |
|
7 |
+
<!-- <p align="center"> <img src="imgs/teaser.jpg" width="100%"> </p> -->
|
8 |
+
|
9 |
+
<table class="center">
|
10 |
+
<tr>
|
11 |
+
<td style="text-align:center;"><b>Input</b></td>
|
12 |
+
<td style="text-align:center;"><b>Output</b></td>
|
13 |
+
<td style="text-align:center;"><b>Input</b></td>
|
14 |
+
<td style="text-align:center;"><b>Output</b></td>
|
15 |
+
</tr>
|
16 |
+
<tr>
|
17 |
+
<td><img src="imgs/obama.jpg"></td>
|
18 |
+
<td><img src="vis_output/obama.jpg"></td>
|
19 |
+
<td><img src="imgs/trump.jpg"></td>
|
20 |
+
<td><img src="vis_output/trump.jpg"></td>
|
21 |
+
</tr>
|
22 |
+
<tr>
|
23 |
+
<td width=25% style="text-align:center;color:gray;">"Who was the president of the US in this image? Please output segmentation mask and explain the reason."</td>
|
24 |
+
<td width=25% style="text-align:center;">"Sure, the segmentation result is [SEG]. The President of the United States in the image is President Obama.”</td>
|
25 |
+
<td width=25% style="text-align:center;color:gray">"Who was the president of the US in this image? Please output segmentation mask and explain why."</td>
|
26 |
+
<td width=25% style="text-align:center;">"Sure, [SEG]. In the image, the President of the United States is President Trump."</td>
|
27 |
+
</tr>
|
28 |
+
|
29 |
+
<tr>
|
30 |
+
<td><img src="imgs/dog_with_horn.jpg"></td>
|
31 |
+
<td><img src="vis_output/dog_with_horn.jpg"></td>
|
32 |
+
<td><img src="imgs/camera_lens.jpg"></td>
|
33 |
+
<td><img src="vis_output/camera_lens.jpg"></td>
|
34 |
+
</tr>
|
35 |
+
<tr>
|
36 |
+
<td width=25% style="text-align:center;color:gray;">"Can you segment the unusual part in this image and explain why."</td>
|
37 |
+
<td width=25% style="text-align:center;">"Sure, it is [SEG]. In the image, the unusual part is the dog wearing a reindeer antler headband. This is not a common sight, as dogs typically do not wear accessories like this..."</td>
|
38 |
+
<td width=25% style="text-align:center;color:gray">"Can you segment the camera lens that is more suitable for photographing nearby objects in this image?"</td>
|
39 |
+
<td width=25% style="text-align:center;">"Sure, [SEG]."</td>
|
40 |
+
</tr>
|
41 |
+
|
42 |
+
<tr>
|
43 |
+
<td><img src="imgs/stand_higher.jpg"></td>
|
44 |
+
<td><img src="vis_output/stand_higher.jpg"></td>
|
45 |
+
<td><img src="imgs/wash_hands.jpg"></td>
|
46 |
+
<td><img src="vis_output/wash_hands.jpg"></td>
|
47 |
+
</tr>
|
48 |
+
<tr>
|
49 |
+
<td width=25% style="text-align:center;color:gray;">"What can make the woman stand higher? Please output segmentation mask and explain why."</td>
|
50 |
+
<td width=25% style="text-align:center;">"Sure, [SEG]. The woman is standing higher by using a ladder. She is on the ladder, which is placed against the side of the house, allowing her to reach the higher parts of the building. This is a common tool used for working on elevated surfaces, such as painting or repairing the exterior of a house."</td>
|
51 |
+
<td width=25% style="text-align:center;color:gray">"Where to wash hands in this image? Please output segmentation mask."</td>
|
52 |
+
<td width=25% style="text-align:center;">"Sure, [SEG]."</td>
|
53 |
+
</tr>
|
54 |
+
</table>
|
55 |
|
56 |
<p align="center"> <img src="imgs/fig_overview.jpg" width="100%"> </p>
|
57 |
|
|
|
|
|
58 |
## News
|
59 |
- [x] [2023.8.4] [Online Demo](http://103.170.5.190:7860/) is released!
|
60 |
- [x] [2023.8.4] [*ReasonSeg* Dataset](https://drive.google.com/drive/folders/125mewyg5Ao6tZ3ZdJ-1-E3n04LGVELqy?usp=sharing) and the [LISA-13B-llama2-v0-explainatory](https://huggingface.co/xinlai/LISA-13B-llama2-v0-explainatory) model are released!
|
vis_output/dog_with_horn.jpg
ADDED
![]() |
vis_output/example1_mask_0.jpg
CHANGED
![]() |
![]() |
vis_output/example1_masked_img_0.jpg
CHANGED
![]() |
![]() |
vis_output/example2_mask_0.jpg
DELETED
Binary file (13.8 kB)
|
|
vis_output/example2_masked_img_0.jpg
DELETED
Binary file (182 kB)
|
|