Update README.md
Browse files
README.md
CHANGED
@@ -1,4 +1,15 @@
|
|
1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2 |
|
3 |
## Examples
|
4 |
|
|
|
1 |
|
2 |
+
# LDM3D-VR model
|
3 |
+
|
4 |
+
The LDM3D model was proposed in ["LDM3D-VR: Latent Diffusion Model for 3D"](https://arxiv.org/pdf/2311.03226.pdf) by Gabriela Ben Melech Stan, Diana Wofk, Estelle Aflalo, Shao-Yen Tseng, Zhipeng Cai, Michael Paulitsch, Vasudev Lal.
|
5 |
+
|
6 |
+
LDM3D got accepted to [NeurIPS Workshop'23 on Diffusion Models][https://neurips.cc/virtual/2023/workshop/66539].
|
7 |
+
|
8 |
+
This new checkpoint related to the upscaler called LDM3D-sr.
|
9 |
+
|
10 |
+
# Model description
|
11 |
+
The abstract from the paper is the following: Latent diffusion models have proven to be state-of-the-art in the creation and manipulation of visual outputs. However, as far as we know, the generation of depth maps jointly with RGB is still limited. We introduce LDM3D-VR, a suite of diffusion models targeting virtual reality development that includes LDM3D-pano
|
12 |
+
and LDM3D-SR. These models enable the generation of panoramic RGBD based on textual prompts and the upscaling of low-resolution inputs to high-resolution RGBD, respectively. Our models are fine-tuned from existing pretrained models on datasets containing panoramic/high-resolution RGB images, depth maps and captions. Both models are evaluated in comparison to existing related methods.
|
13 |
|
14 |
## Examples
|
15 |
|