metadata

license: apache-2.0
datasets:
  - KwaiVGI/360Motion-Dataset
tags:
  - 3D controllable video generation

3DTrajMaster-CogVideoX

Introduction

3DTrajMaster controls one or multiple entity motions in 3D space with entity-specific 3D trajectories for text-to-video (T2V) generation. It has the following features::

6 Domain of Freedom (DoF): control 3D entity location and orientation.
Diverse Entities: human, animal, robot, car, even abstract fire, breeze, etc.
Diverse Background: city, forest, desert, gym, sunset beach, glacier, hall, night city, etc.
Complex 3D trajectories: 3D occlusion, rotating in place, 180°/continuous 90° turnings, etc.
Fine-grained Entity Prompt: change human hair, clothing, gender, figure size, accessory, etc.

Usage

This is the implementation based on CogVideoX-5B. Please refer to our github for details on usage.

Citation

If you find this project useful, please consider citing:

@inproceedings{fu20243dtrajmaster,
    author    = {Fu, Xiao and Liu, Xian and Wang, Xintao and Peng, Sida and Xia, Menghan and Shi, Xiaoyu and Yuan, Ziyang and Wan, Pengfei and Zhang, Di and Lin, Dahua},
    title     = {3DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generation},
    booktitle = {ICLR},
    year      = {2025}
}