README.md

[CVPR 2025] AvatarArtist: Open-Domain 4D Avatarization

If you like our project, please give us a star ⭐ on GitHub for the latest update.

This repository contains the official implementation of AvatarArtist, a method for generating 4D avatars from a single image.

💡 We also have other avatar projects that may interest you ✨.

HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillation, SIGGRAPH 2024
Hongyu Liu, Xuan Wang, Ziyu Wan, etc.

Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation, SIGGRAPH Asia 2024
Yue Ma, Hongyu Liu, Hongfa Wang, etc.

🚨 News

• [03/26/2025] Inference Code and pretrained models released!

⚙️ Setup

Environment

git clone --depth=1 https://github.com/ant-research/AvatarArtist 
cd AvatarArtist
conda create -n avatarartist python=3.9.0
conda activate avatarartist
pip install -r requirements.txt

Download Weights

The weights are available at 🤗HuggingFace, you can download it with the following commands. Please move the required files into the pretrained_model directory:

# if you are in china mainland, run this first: export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download --repo-type model \
KUMAPOWER/AvatarArtist \
--local-dir pretrained_model

🤗 Usage

Inference

Our approach consists of two steps during the inference process. First, the DiT model generates a 4D representation based on the input image. Then, our Motion-Aware Cross-Domain Renderer takes this 4D representation as input and, guided by both the input image and driving signals, renders it into the final target image.

This is an example of inference using the demo data. The images used in this example are sourced from https://civitai.com/.

python3 inference.py \
    --img_file './demo_data/source_img/img_from_web/images512x512/final_ipimgs' \
    --input_img_fvid './demo_data/source_img/img_from_web/coeffs/final_ipimgs' \
    --input_img_motion './demo_data/source_img/img_from_web/motions/final_ipimgs' \
    --video_name 'Obama' \
    --target_path './demo_data/target_video/data_obama'
    # --use_demo_cam (create a video like the teaser using predefined camera parameters)

This is an example of performing inference using the model. The images used in this example are diverse-domain images generated by a diffusion model, as described in our paper. You can use the --select_img option to specify a particular input image.

python3 inference.py \
    --img_file './demo_data/source_img/img_generate_different_domain/images512x512/demo_imgs' \
    --input_img_fvid './demo_data/img_generate_different_domain/coeffs/demo_imgs' \
    --input_img_motion './demo_data/source_img/img_generate_different_domain/motions/demo_imgs' \
    --video_name "Obama" \
    --target_path './demo_data/target_video/data_obama' \
    --select_img 'your_selected_image.png in img_file'

Custom Data Processing

We provide a set of scripts to process input images and videos for use with our model. These scripts ensure that the data is properly formatted and preprocessed, making it compatible with our inference pipeline. You can use them to prepare your own custom data for generating results with our model.

Please refer to this guide to learn how to obtain the inference data. You can also check the demo data for reference. The data structure is shown below.

The files in the "dataset" folder serve as the final input to the model, while the other files are intermediate outputs from the data processing pipeline：

📦 datasets/
├── 📂 dataset/
│   ├── 📂 coeffs/
│   ├── 📂 images512x512/
│   ├── 📂 uvRender256x256/
│   ├── 📂 orthRender256x256_face_eye/
│   ├── 📂 motions/
├── 📂 crop_fv_tracking/
├── 📂 realign_detections/
├── 📂 realign_detections/
├── 📂 realign/
├── 📂 raw_detection/
├── 📂 align_3d_landmark/
├── 📂 raw_frames/

Different domain's input images generation

We provide a set of scripts to transfer the realistic domain's portrait to the other domain. Please refer to this guide.

📋 To-Do List

Pending Tasks

• Gradio demo
• Release training code

---

✅ Completed Tasks

• Release inference code
• Release data processing tools
• Release the pipeline to generate input for different domains

👍 Credits

We sincerely appreciate the contributions of the following open-source projects, which have significantly influenced our work:

• DiT builds upon PixArt-alpha.
• VAE is based on LVDM.
• Motion-aware rendering is inspired by Portrait4D.
• 4D representation in our paper is proposed in Next3D and Next3D++.
• We referenced DATID3D for domain-specific prompts.

🔒 License

• The majority of this project is released under the Apache 2.0 license as found in the LICENSE file.

✏️ Citation

If you make use of our work, please cite our paper.

@article{liu2025avatarartist,
  title={AvatarArtist: Open-Domain 4D Avatarization},
  author={Hongyu Liu, Xuan Wang, Ziyu Wan, Yue Ma, Jingye Chen, Yanbo Fan, Yujun Shen, Yibing Song, Qifeng Chen },
  booktitle={CVPR},
  year={2025}
}