Update README.md

README.md

@@ -1,3 +1,92 @@
 ---
 license: apache-2.0
+language:
+- en
 ---
+
+# MotionCtrl Model Card
+
+<div align="center">
+
+[**Project Page**](https://wzhouxiff.github.io/projects/MotionCtrl/) **|** [**Paper (ArXiv)**](https://arxiv.org/pdf/2312.03641.pdf) **|** [**Code**](https://github.com/TencentARC/MotionCtrl)
+
+[🤗 **Gradio demo (MotionCtrl+VideoCrafter)**](https://huggingface.co/spaces/TencentARC/MotionCtrl) **|** [🤗 **Gradio demo (MotionCtrl+SVD)**](https://huggingface.co/spaces/TencentARC/MotionCtrl_SVD)
+
+</div>
+
+## Introduction
+
+MotionCtrl, when provided with specific camera movements indicated by camera poses or several trajectories, 
+can independently control both the **camera** and **object** motion in a generated video. 
+It can be deployed on both Text-to-Video generation models, such as **LVDM/VideoCrafter** and **AnimateDiff**, 
+as well as Image-to-Video generation models like **SVD**.
+
+## MotionCtrl + SVD
+
+- Model: [motionctrl.pth](https://huggingface.co/TencentARC/MotionCtrl/blob/main/motionctrl.pth)
+- Results: 
+
+<div align="center">
+    <img src="https://raw.githubusercontent.com/TencentARC/MotionCtrl/main/assets/svd/00_ibzz5-dxv2h.gif", width="49.7%", height="100%", style="float: left"> 
+    <img src="https://raw.githubusercontent.com/TencentARC/MotionCtrl/main/assets/svd/01_5guvn-0x6v2.gif", width="49.7%", height="100%", style="float: right">
+</div>
+
+&ensp;
+
+## MotionCtrl + LVDM/VideoCrafter
+
+- Model: [motionctrl_svd.ckpt](https://huggingface.co/TencentARC/MotionCtrl/blob/main/motionctrl_svd.ckpt)
+- Results:
+
+<div align="center">
+    <img src="https://raw.githubusercontent.com/TencentARC/MotionCtrl/main/assets/hpxvu-3d8ym.gif", width="49.7%", height="100%", style="float: left">
+    <img src="https://raw.githubusercontent.com/TencentARC/MotionCtrl/main/assets/w3nb7-9vz5t.gif", width="49.7%", height="100%", style="float: right">
+    <img src="https://raw.githubusercontent.com/TencentARC/MotionCtrl/main/assets/62n2a-wuvsw.gif", width="49.7%", height="100%", style="float: left">
+    <img src="https://raw.githubusercontent.com/TencentARC/MotionCtrl/main/assets/ilw96-ak827.gif", width="49.7%", height="100%", style="float: right">
+</div>
+
+&ensp;
+
+## MotionCtrl + AnimateDiff
+
+- Model: [Coming Soon]()
+- Results:
+
+<div align="center">
+    <img src="https://raw.githubusercontent.com/wzhouxiff/wzhouxiff.github.io/master/projects/MotionCtrl/assets/videos/ours_animatediff/complex/1424acd0007d40b5.png", width="25%", height="100%", style="float: left">
+    <img src="https://raw.githubusercontent.com/wzhouxiff/wzhouxiff.github.io/master/projects/MotionCtrl/assets/videos/ours_animatediff/complex/1424acd0007d40b5.gif", width="25%", height="100%", style="float: left">
+    <img src="https://raw.githubusercontent.com/wzhouxiff/wzhouxiff.github.io/master/projects/MotionCtrl/assets/videos/ours_animatediff/complex/camera_Round-R_ZoomIn.png", width="25%", height="100%", style="float: left">
+    <img src="https://raw.githubusercontent.com/wzhouxiff/wzhouxiff.github.io/master/projects/MotionCtrl/assets/videos/ours_animatediff/complex/round_in.gif", width="25%", height="100%", style="float: left">
+</div>
+
+<div align="center">
+    <img src="https://raw.githubusercontent.com/wzhouxiff/wzhouxiff.github.io/master/projects/MotionCtrl/assets/videos/ours_animatediff/traj/traj_curve_1.gif", width="25%", height="100%", style="float: left">
+    <img src="https://raw.githubusercontent.com/wzhouxiff/wzhouxiff.github.io/master/projects/MotionCtrl/assets/videos/ours_animatediff/traj/curve_1_1.gif", width="25%", height="100%", style="float: left">
+    <img src="https://raw.githubusercontent.com/wzhouxiff/wzhouxiff.github.io/master/projects/MotionCtrl/assets/videos/ours_animatediff/traj/curve_1_2.gif", width="25%", height="100%", style="float: left">
+    <img src="https://raw.githubusercontent.com/wzhouxiff/wzhouxiff.github.io/master/projects/MotionCtrl/assets/videos/ours_animatediff/traj/curve_1.gif", width="25%", height="100%", style="float: left">
+</div>
+
+&ensp;
+
+## Usage
+
+- Download directly in this repository OR Download with python script
+
+```python
+from huggingface_hub import hf_hub_download
+motionctrl_lvdm_ckpt = hf_hub_download(repo_id="TencentARC/MotionCtrl", filename="motionctrl.ckpt", repo_type="model")
+motionctrl_svd_ckpt = hf_hub_download(repo_id="TencentARC/MotionCtrl", filename="motionctrl_svd.ckpt", repo_type="model")
+```
+
+- Then generate the controllable video following the instructions in our [GitHub repository](https://github.com/TencentARC/MotionCtrl/).
+
+## Citation
+
+```bibtex
+@inproceedings{wang2023motionctrl,
+  title={MotionCtrl: A Unified and Flexible Motion Controller for Video Generation},
+  author={Wang, Zhouxia and Yuan, Ziyang and Wang, Xintao and Chen, Tianshui and Xia, Menghan and Luo, Ping and Shan, Yin},
+  booktitle={arXiv preprint arXiv:2312.03641},
+  year={2023}
+}
+```