update README
Browse files
README.md
CHANGED
@@ -1,3 +1,48 @@
|
|
1 |
-
---
|
2 |
-
license:
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: cc-by-nc-sa-4.0
|
3 |
+
pipeline_tag: Image-to-Video
|
4 |
+
tags:
|
5 |
+
- turing
|
6 |
+
- autonomous driving
|
7 |
+
- video generation
|
8 |
+
- world model
|
9 |
+
---
|
10 |
+
|
11 |
+
# Terra
|
12 |
+
|
13 |
+
**Terra** is a world model designed for autonomous driving and serves as a baseline model in th [ACT-Bench](https://github.com/turingmotors/ACT-Bench) framework.
|
14 |
+
Terra generates video continuations based on short video clips of approximately three frames and trajectory instructions.
|
15 |
+
A key feature of Terra is its **high adherence to trajectory instructions**, enabling accurate and reliable action-conditioned video generation.
|
16 |
+
|
17 |
+
## Related Links
|
18 |
+
|
19 |
+
For more technical details and discussions, please refer to:
|
20 |
+
- **Paper:** To Be Updated
|
21 |
+
- **Code:** https://github.com/turingmotors/ACT-Bench
|
22 |
+
- **Blog Post:** [運転版の"Sora"を作る:動画生成の世界モデルTerraの開発背景](https://zenn.dev/turing_motors/articles/6c0ddc10aae542) (ja) / [Create a driving version of "Sora"](https://medium.com/@hide1996/create-a-driving-version-of-sora-33cf4040937a) (en)
|
23 |
+
|
24 |
+
## How to use
|
25 |
+
|
26 |
+
We have verified the execution on a machine equipped with a single NVIDIA H100 80GB GPU. However, we believe it should be possible to run the model on any machine equipped with an NVIDIA GPU with 16GB or more of VRAM.
|
27 |
+
|
28 |
+
### Install Packages
|
29 |
+
|
30 |
+
|
31 |
+
|
32 |
+
### Action-Conditioned Video Generation without Video Refiner
|
33 |
+
|
34 |
+
### Action-Conditioned Video Generation with Video Refiner
|
35 |
+
|
36 |
+
## Citation
|
37 |
+
|
38 |
+
```bibtex
|
39 |
+
@misc{arai2024actbench,
|
40 |
+
title={ACT-Bench: Towards Action Controllable World Models for Autonomous Driving},
|
41 |
+
author={Hidehisa Arai and Keishi Ishihara and Tsubasa Takahashi and Yu Yamaguchi},
|
42 |
+
year={2024},
|
43 |
+
eprint={2412.05337},
|
44 |
+
archivePrefix={arXiv},
|
45 |
+
primaryClass={cs.CV},
|
46 |
+
url={https://arxiv.org/abs/2412.05337},
|
47 |
+
}
|
48 |
+
```
|