Update README.md
Browse files
README.md
CHANGED
@@ -21,15 +21,21 @@ Terminal Technology Department, Alipay, Ant Group.
|
|
21 |
<a href='https://arxiv.org/abs/2411.10061'><img src='https://img.shields.io/badge/Paper-Arxiv-red'></a>
|
22 |
<a href='https://github.com/antgroup/echomimic_v2/blob/main/assets/halfbody_demo/wechat_group.png'><img src='https://badges.aleen42.com/src/wechat.svg'></a>
|
23 |
</div>
|
|
|
|
|
|
|
|
|
24 |
|
25 |
## 🚀 EchoMimic Series
|
26 |
* EchoMimicV1: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditioning. [GitHub](https://github.com/antgroup/echomimic)
|
27 |
* EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation. [GitHub](https://github.com/antgroup/echomimic_v2)
|
28 |
|
29 |
## 📣 Updates
|
|
|
|
|
30 |
* [2024.11.22] ๐ฅ [ComfyUI](https://github.com/smthemex/ComfyUI_EchoMimic) is now available. Thanks @smthemex for the contribution.
|
31 |
-
* [2024.11.
|
32 |
-
* [2024.11.
|
33 |
* [2024.11.15] ๐ฅ Our [paper](https://arxiv.org/abs/2411.10061) is in public on arxiv.
|
34 |
|
35 |
## 🌅 Gallery
|
@@ -149,7 +155,11 @@ Create conda environment (Recommended):
|
|
149 |
|
150 |
Install packages with `pip`
|
151 |
```bash
|
|
|
|
|
|
|
152 |
pip install -r requirements.txt
|
|
|
153 |
```
|
154 |
|
155 |
### Download ffmpeg-static
|
@@ -178,7 +188,7 @@ The **pretrained_weights** is organized as follows.
|
|
178 |
โโโ sd-image-variations-diffusers
|
179 |
โ โโโ ...
|
180 |
โโโ audio_processor
|
181 |
-
โโโ
|
182 |
```
|
183 |
|
184 |
In which **denoising_unet.pth** / **reference_unet.pth** / **motion_module.pth** / **pose_encoder.pth** are the main checkpoints of **EchoMimic**. Other models in this hub can be also downloaded from it's original hub, thanks to their brilliant works:
|
@@ -187,6 +197,10 @@ In which **denoising_unet.pth** / **reference_unet.pth** / **motion_module.pth**
|
|
187 |
- [audio_processor(whisper)](https://openaipublic.azureedge.net/main/whisper/models/65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9/tiny.pt)
|
188 |
|
189 |
### Inference on Demo
|
|
|
|
|
|
|
|
|
190 |
Run the python inference script:
|
191 |
```bash
|
192 |
python infer.py --config='./configs/prompts/infer.yaml'
|
@@ -245,4 +259,4 @@ If you find our work useful for your research, please consider citing the paper
|
|
245 |
```
|
246 |
|
247 |
## 🌟 Star History
|
248 |
-
[![Star History Chart](https://api.star-history.com/svg?repos=antgroup/echomimic_v2&type=Date)](https://star-history.com/#antgroup/echomimic_v2&Date)
|
|
|
21 |
<a href='https://arxiv.org/abs/2411.10061'><img src='https://img.shields.io/badge/Paper-Arxiv-red'></a>
|
22 |
<a href='https://github.com/antgroup/echomimic_v2/blob/main/assets/halfbody_demo/wechat_group.png'><img src='https://badges.aleen42.com/src/wechat.svg'></a>
|
23 |
</div>
|
24 |
+
<div align='center'>
|
25 |
+
<a href='https://github.com/antgroup/echomimic_v2/discussions/53'><img src='https://img.shields.io/badge/English-Common Problems-orange'></a>
|
26 |
+
<a href='https://github.com/antgroup/echomimic_v2/discussions/40'><img src='https://img.shields.io/badge/ไธญๆ็-ๅธธ่ง้ฎ้ขๆฑๆป-orange'></a>
|
27 |
+
</div>
|
28 |
|
29 |
## 🚀 EchoMimic Series
|
30 |
* EchoMimicV1: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditioning. [GitHub](https://github.com/antgroup/echomimic)
|
31 |
* EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation. [GitHub](https://github.com/antgroup/echomimic_v2)
|
32 |
|
33 |
## 📣 Updates
|
34 |
+
* [2024.11.27] ๐ฅ Thanks [AiMotionStudio](https://www.youtube.com/@AiMotionStudio) for the [installation tutorial](https://www.youtube.com/watch?v=2ab6U1-nVTQ).
|
35 |
+
* [2024.11.22] ๐ฅ [GradioUI](https://github.com/antgroup/echomimic_v2/blob/main/app.py) is now available. Thanks @gluttony-10 for the contribution.
|
36 |
* [2024.11.22] ๐ฅ [ComfyUI](https://github.com/smthemex/ComfyUI_EchoMimic) is now available. Thanks @smthemex for the contribution.
|
37 |
+
* [2024.11.21] ๐ฅ We release the EMTD dataset list and processing scripts.
|
38 |
+
* [2024.11.21] ๐ฅ We release our [EchoMimicV2](https://github.com/antgroup/echomimic_v2) codes and models.
|
39 |
* [2024.11.15] ๐ฅ Our [paper](https://arxiv.org/abs/2411.10061) is in public on arxiv.
|
40 |
|
41 |
## 🌅 Gallery
|
|
|
155 |
|
156 |
Install packages with `pip`
|
157 |
```bash
|
158 |
+
pip install pip -U
|
159 |
+
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 xformers==0.0.28.post3 --index-url https://download.pytorch.org/whl/cu124
|
160 |
+
pip install torchao --index-url https://download.pytorch.org/whl/nightly/cu124
|
161 |
pip install -r requirements.txt
|
162 |
+
pip install --no-deps facenet_pytorch==2.6.0
|
163 |
```
|
164 |
|
165 |
### Download ffmpeg-static
|
|
|
188 |
โโโ sd-image-variations-diffusers
|
189 |
โ โโโ ...
|
190 |
โโโ audio_processor
|
191 |
+
โโโ tiny.pt
|
192 |
```
|
193 |
|
194 |
In which **denoising_unet.pth** / **reference_unet.pth** / **motion_module.pth** / **pose_encoder.pth** are the main checkpoints of **EchoMimic**. Other models in this hub can be also downloaded from it's original hub, thanks to their brilliant works:
|
|
|
197 |
- [audio_processor(whisper)](https://openaipublic.azureedge.net/main/whisper/models/65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9/tiny.pt)
|
198 |
|
199 |
### Inference on Demo
|
200 |
+
Run the gradio:
|
201 |
+
```bash
|
202 |
+
python app.py
|
203 |
+
```
|
204 |
Run the python inference script:
|
205 |
```bash
|
206 |
python infer.py --config='./configs/prompts/infer.yaml'
|
|
|
259 |
```
|
260 |
|
261 |
## 🌟 Star History
|
262 |
+
[![Star History Chart](https://api.star-history.com/svg?repos=antgroup/echomimic_v2&type=Date)](https://star-history.com/#antgroup/echomimic_v2&Date)
|