therutvikpanchal
commited on
Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,134 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: mit
|
3 |
+
pipeline_tag: image-segmentation
|
4 |
+
tags:
|
5 |
+
- remove background
|
6 |
+
- background
|
7 |
+
- background-removal
|
8 |
+
- Pytorch
|
9 |
+
- vision
|
10 |
+
- legal liability
|
11 |
+
- transformers
|
12 |
+
---
|
13 |
+
|
14 |
+
# BRIA Background Removal v2.0 Model Card
|
15 |
+
|
16 |
+
RMBG v2.0 is our new state-of-the-art background removal model significantly improves RMBG v1.4. The model is designed to effectively separate foreground from background in a range of
|
17 |
+
categories and image types. This model has been trained on a carefully selected dataset, which includes:
|
18 |
+
general stock images, e-commerce, gaming, and advertising content, making it suitable for commercial use cases powering enterprise content creation at scale.
|
19 |
+
The accuracy, efficiency, and versatility currently rival leading source-available models.
|
20 |
+
It is ideal where content safety, legally licensed datasets, and bias mitigation are paramount.
|
21 |
+
|
22 |
+
Developed by BRIA AI, RMBG v2.0 is available as a source-available model for non-commercial use.
|
23 |
+
|
24 |
+
## Model Details
|
25 |
+
#####
|
26 |
+
### Model Description
|
27 |
+
|
28 |
+
- **Developed by:** [BRIA AI](https://bria.ai/)
|
29 |
+
- **Model type:** Background Removal
|
30 |
+
|
31 |
+
- **Model Description:** BRIA RMBG-2.0 is a dichotomous image segmentation model trained exclusively on a professional-grade dataset.
|
32 |
+
- **BRIA:** Resources for more information: [BRIA AI](https://bria.ai/)
|
33 |
+
|
34 |
+
|
35 |
+
|
36 |
+
## Training data
|
37 |
+
Bria-RMBG model was trained with over 15,000 high-quality, high-resolution, manually labeled (pixel-wise accuracy), fully licensed images.
|
38 |
+
Our benchmark included balanced gender, balanced ethnicity, and people with different types of disabilities.
|
39 |
+
For clarity, we provide our data distribution according to different categories, demonstrating our model’s versatility.
|
40 |
+
|
41 |
+
### Distribution of images:
|
42 |
+
|
43 |
+
| Category | Distribution |
|
44 |
+
| -----------------------------------| -----------------------------------:|
|
45 |
+
| Objects only | 45.11% |
|
46 |
+
| People with objects/animals | 25.24% |
|
47 |
+
| People only | 17.35% |
|
48 |
+
| people/objects/animals with text | 8.52% |
|
49 |
+
| Text only | 2.52% |
|
50 |
+
| Animals only | 1.89% |
|
51 |
+
|
52 |
+
| Category | Distribution |
|
53 |
+
| -----------------------------------| -----------------------------------------:|
|
54 |
+
| Photorealistic | 87.70% |
|
55 |
+
| Non-Photorealistic | 12.30% |
|
56 |
+
|
57 |
+
|
58 |
+
| Category | Distribution |
|
59 |
+
| -----------------------------------| -----------------------------------:|
|
60 |
+
| Non Solid Background | 52.05% |
|
61 |
+
| Solid Background | 47.95%
|
62 |
+
|
63 |
+
|
64 |
+
| Category | Distribution |
|
65 |
+
| -----------------------------------| -----------------------------------:|
|
66 |
+
| Single main foreground object | 51.42% |
|
67 |
+
| Multiple objects in the foreground | 48.58% |
|
68 |
+
|
69 |
+
|
70 |
+
## Qualitative Evaluation
|
71 |
+
Open source models comparison
|
72 |
+
![diagram](diagram1.png)
|
73 |
+
![examples](collage5.png)
|
74 |
+
|
75 |
+
### Architecture
|
76 |
+
RMBG-2.0 is developed on the [BiRefNet](https://github.com/ZhengPeng7/BiRefNet) architecture enhanced with our proprietary dataset and training scheme. This training data significantly improves the model’s accuracy and effectiveness for background-removal task.<br>
|
77 |
+
If you use this model in your research, please cite:
|
78 |
+
|
79 |
+
```
|
80 |
+
@article{BiRefNet,
|
81 |
+
title={Bilateral Reference for High-Resolution Dichotomous Image Segmentation},
|
82 |
+
author={Zheng, Peng and Gao, Dehong and Fan, Deng-Ping and Liu, Li and Laaksonen, Jorma and Ouyang, Wanli and Sebe, Nicu},
|
83 |
+
journal={CAAI Artificial Intelligence Research},
|
84 |
+
year={2024}
|
85 |
+
}
|
86 |
+
```
|
87 |
+
|
88 |
+
#### Requirements
|
89 |
+
```bash
|
90 |
+
torch
|
91 |
+
torchvision
|
92 |
+
pillow
|
93 |
+
kornia
|
94 |
+
transformers
|
95 |
+
```
|
96 |
+
|
97 |
+
### Usage
|
98 |
+
|
99 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
100 |
+
|
101 |
+
|
102 |
+
```python
|
103 |
+
from PIL import Image
|
104 |
+
import matplotlib.pyplot as plt
|
105 |
+
import torch
|
106 |
+
from torchvision import transforms
|
107 |
+
from transformers import AutoModelForImageSegmentation
|
108 |
+
|
109 |
+
model = AutoModelForImageSegmentation.from_pretrained('briaai/RMBG-2.0', trust_remote_code=True)
|
110 |
+
torch.set_float32_matmul_precision(['high', 'highest'][0])
|
111 |
+
model.to('cuda')
|
112 |
+
model.eval()
|
113 |
+
|
114 |
+
# Data settings
|
115 |
+
image_size = (1024, 1024)
|
116 |
+
transform_image = transforms.Compose([
|
117 |
+
transforms.Resize(image_size),
|
118 |
+
transforms.ToTensor(),
|
119 |
+
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
|
120 |
+
])
|
121 |
+
|
122 |
+
image = Image.open(input_image_path)
|
123 |
+
input_images = transform_image(image).unsqueeze(0).to('cuda')
|
124 |
+
|
125 |
+
# Prediction
|
126 |
+
with torch.no_grad():
|
127 |
+
preds = model(input_images)[-1].sigmoid().cpu()
|
128 |
+
pred = preds[0].squeeze()
|
129 |
+
pred_pil = transforms.ToPILImage()(pred)
|
130 |
+
mask = pred_pil.resize(image.size)
|
131 |
+
image.putalpha(mask)
|
132 |
+
|
133 |
+
image.save("no_bg_image.png")
|
134 |
+
```
|