File size: 2,229 Bytes
a714244 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
# Preparation
This is the official implementation of *LucidFusion: Generating 3D Gaussians with Arbitrary Unposed Images*.
## Install
```
# Xformers is required! please refer to https://github.com/facebookresearch/xformers for details.
# For example, we use torch 2.3.1 + cuda 11.8
conda install pytorch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 pytorch-cuda=11.8 -c pytorch -c nvidia
# [linux only] cuda 11.8 version
pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu118
# For 3D Gaussian Splatting, we use the official installation. Please refer to https://github.com/graphdeco-inria/gaussian-splatting for details
git clone https://github.com/graphdeco-inria/gaussian-splatting --recursive
pip install ./diff-gaussian-rasterization
# Other dependencies
conda env create -f environment.yml
conda activate LucidFusion
```
## Pretrained Weights
Our pre-trained weights will be released soon, please check back!
Our current model loads pre-trained diffusion model weights ([Stable Diffusion 2.1] (https://huggingface.co/stabilityai/stable-diffusion-2-1/tree/main) in default) for training purposes.
## Inference
A shell script is provided with example files.
To run, you first need to setup the pretrained weights as follows:
```
cd LucidFusion
mkdir output/demo
# Download the pretrained weights and name it as best.ckpt
# Place the pretrained weights in LucidFusion/output/demo/best.ckpt
# The result will be saved in the LucidFusion/output/demo/<obj name>/
bash scripts/demo.sh
```
More example files will be released soon
You can also try your own example! To do that:
First obtain images and place them in the examples folder
```
LucidFusion
├── examples/
| ├── "your obj name"/
| | ├── "image_01.png"
| | ├── "image_02.png"
| | ├── ...
```
Then you need to run preprocess.py to extract recentered image and its mask:
```
# Run the following will create two folders (images, masks) in "your obj name" folder.
# You can check to see if the extract mask is corrected.
python preprocess.py "examples/"you obj name" --outdir examples/"your obj name"
# Modify demo.sh to DEMO='examples/"you obj name" ' then run the file
bash scripts/demo.sh
```
|