XCLiu commited on
Commit
495fa9d
1 Parent(s): 08d715b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -20
README.md CHANGED
@@ -18,12 +18,6 @@ JanusFlow introduces a minimalist architecture that integrates autoregressive
18
  language models with rectified flow, a state-of-the-art method in generative modeling. Our
19
  key finding demonstrates that rectified flow can be straightforwardly trained within the large
20
  language model framework, eliminating the need for complex architectural modifications.
21
- To further improve the performance of our unified model, we adopt two key strategies: (i)
22
- decoupling the understanding and generation encoders, and (ii) aligning their representations
23
- during unified training. Extensive experiments show that JanusFlow achieves comparable or
24
- superior performance to specialized models in their respective domains, while significantly
25
- outperforming existing unified approaches across standard benchmarks. This work represents
26
- a step toward more efficient and versatile vision-language models.
27
 
28
  [JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation](TBD)
29
 
@@ -36,12 +30,12 @@ a step toward more efficient and versatile vision-language models.
36
 
37
  ### 2. Model Summary
38
 
39
- Janus is a unified understanding and generation MLLM, which decouples visual encoding for multimodal understanding and generation.
40
- Janus is constructed based on the DeepSeek-LLM-1.3b-base which is trained on an approximate corpus of 500B text tokens.
41
- For multimodal understanding, it uses the [SigLIP-L](https://huggingface.co/timm/ViT-L-16-SigLIP-384) as the vision encoder, which supports 384 x 384 image input. For image generation, Janus uses the tokenizer from [here](https://github.com/FoundationVision/LlamaGen) with a downsample rate of 16.
42
 
43
  <div align="center">
44
- <img alt="image" src="arch.jpg" style="width:90%;">
45
  </div>
46
 
47
  ## 3. Quick Start
@@ -51,19 +45,12 @@ Please refer to [**Github Repository**](https://github.com/deepseek-ai/Janus)
51
 
52
  ## 4. License
53
 
54
- This code repository is licensed under [the MIT License](https://github.com/deepseek-ai/DeepSeek-LLM/blob/HEAD/LICENSE-CODE). The use of Janus models is subject to [DeepSeek Model License](https://github.com/deepseek-ai/DeepSeek-LLM/blob/HEAD/LICENSE-MODEL).
 
55
  ## 5. Citation
56
 
57
  ```
58
- @misc{wu2024janus,
59
- title={Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation},
60
- author={Chengyue Wu and Xiaokang Chen and Zhiyu Wu and Yiyang Ma and Xingchao Liu and Zizheng Pan and Wen Liu and Zhenda Xie and Xingkai Yu and Chong Ruan and Ping Luo},
61
- year={2024},
62
- eprint={2410.13848},
63
- archivePrefix={arXiv},
64
- primaryClass={cs.CV},
65
- url={https://arxiv.org/abs/2410.13848},
66
- }
67
  ```
68
 
69
  ## 6. Contact
 
18
  language models with rectified flow, a state-of-the-art method in generative modeling. Our
19
  key finding demonstrates that rectified flow can be straightforwardly trained within the large
20
  language model framework, eliminating the need for complex architectural modifications.
 
 
 
 
 
 
21
 
22
  [JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation](TBD)
23
 
 
30
 
31
  ### 2. Model Summary
32
 
33
+ JanusFlow is a unified understanding and generation MLLM, which decouples visual encoding for multimodal understanding and generation, which is constructed based on DeepSeek-LLM-1.3b-base.
34
+ For multimodal understanding, it uses the [SigLIP-L](https://huggingface.co/timm/ViT-L-16-SigLIP-384) as the vision encoder, which supports 384 x 384 image input.
35
+ For image generation, JanusFlow uses rectified flow to generate in the latent space of [SDXL-VAE](https://huggingface.co/stabilityai/sdxl-vae).
36
 
37
  <div align="center">
38
+ <img alt="image" src="arch.png" style="width:90%;">
39
  </div>
40
 
41
  ## 3. Quick Start
 
45
 
46
  ## 4. License
47
 
48
+ This code repository is licensed under [the MIT License](https://github.com/deepseek-ai/DeepSeek-LLM/blob/HEAD/LICENSE-CODE). The use of JanusFlow models is subject to [DeepSeek Model License](https://github.com/deepseek-ai/DeepSeek-LLM/blob/HEAD/LICENSE-MODEL).
49
+
50
  ## 5. Citation
51
 
52
  ```
53
+ TBD
 
 
 
 
 
 
 
 
54
  ```
55
 
56
  ## 6. Contact