Upload README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,28 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: cc-by-4.0
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-4.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- text-to-video
|
| 7 |
+
---
|
| 8 |
+
# <span style="font-family: 'Courier New', monospace; font-weight: bold">Data-Juicer Sandbox: A Comprehensive Suite for Multimodal Data-Model Co-development
|
| 9 |
+
|
| 10 |
+
## Project description
|
| 11 |
+
|
| 12 |
+
The emergence of large-scale multi-modal generative models has drastically advanced artificial intelligence, introducing unprecedented levels of performance and functionality.
|
| 13 |
+
However, optimizing these models remains challenging due to historically isolated paths of model-centric and data-centric developments, leading to suboptimal outcomes and inefficient resource utilization.
|
| 14 |
+
In response, we present a novel sandbox suite tailored for integrated data-model co-development. This sandbox provides a comprehensive experimental platform, enabling rapid iteration and insight-driven refinement of both data and models.
|
| 15 |
+
Our proposed "Probe-Analyze-Refine" workflow, validated through applications on [T2V-Turbo](https://github.com/Ji4chenLi/t2v-turbo) and achieve a new state-of-the-art on [VBench leaderboard](https://huggingface.co/spaces/Vchitect/VBench_Leaderboard) with 1.09% improvement from T2V-Turbo. Our experiment codes and dataset are released at [Data-Juicer Sandbox](https://github.com/modelscope/data-juicer/blob/main/docs/Sandbox.md).
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
## Model description 🚀
|
| 19 |
+
|
| 20 |
+
This repository contains `unet_lora.pt` that can turn [VideoCrafter2](https://ailab-cvc.github.io/videocrafter2/) into our <span style="font-family: 'Courier New', monospace; font-weight: bold">Data-Juicer-T2V</span>. . Please infer the codes in [T2V-Turbo](https://github.com/Ji4chenLi/t2v-turbo) to infer our model.
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
## Guidelines on Inappropriate and Prohibited Use 🚫
|
| 24 |
+
This model is intended solely for research and educational purposes.
|
| 25 |
+
|
| 26 |
+
- Generating content that could be considered insulting or detrimental to individuals or their surroundings, including their culture, religion, etc., is strictly forbidden.
|
| 27 |
+
- The creation of content that is pornographic, violent, or graphically disturbing is not allowed.
|
| 28 |
+
- Users must not use the model to produce incorrect or misleading information.
|