|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- KwaiVGI/360Motion-Dataset |
|
tags: |
|
- 3D controllable video generation |
|
--- |
|
# 3DTrajMaster-CogVideoX |
|
|
|
## Introduction |
|
3DTrajMaster controls **one or multiple entity motions in 3D space with entity-specific 3D trajectories** for text-to-video (T2V) generation. It has the following features:: |
|
- **6 Domain of Freedom (DoF)**: control 3D entity location and orientation. |
|
- **Diverse Entities**: human, animal, robot, car, even abstract fire, breeze, etc. |
|
- **Diverse Background**: city, forest, desert, gym, sunset beach, glacier, hall, night city, etc. |
|
- **Complex 3D trajectories**: 3D occlusion, rotating in place, 180°/continuous 90° turnings, etc. |
|
- **Fine-grained Entity Prompt**: change human hair, clothing, gender, figure size, accessory, etc. |
|
|
|
|
|
## Usage |
|
|
|
This is the implementation based on CogVideoX-5B. Please refer to our [github](https://github.com/KwaiVGI/3DTrajMaster) for details on usage. |
|
|
|
<video controls autoplay width="100%" src="https://cdn-uploads.huggingface.co/production/uploads/63aef2cafcca84593e6682db/Jxpa_f3nv_pkQM5Knlrp9.mp4"></video> |
|
|
|
## Citation |
|
|
|
If you find this project useful, please consider citing: |
|
|
|
```bibtex |
|
@inproceedings{fu20243dtrajmaster, |
|
author = {Fu, Xiao and Liu, Xian and Wang, Xintao and Peng, Sida and Xia, Menghan and Shi, Xiaoyu and Yuan, Ziyang and Wan, Pengfei and Zhang, Di and Lin, Dahua}, |
|
title = {3DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generation}, |
|
booktitle = {ICLR}, |
|
year = {2025} |
|
} |
