LEGO_StableDiffusion
Overview
The project aim was to fine-tuned stable diffusion model in order to generate images in the LEGO style based on the prompt.Fine-tuned model is created on top of Stable Diffusion text to image model.
To run app install requirements by typing: pip3 install -r requirements, then run the main app file python3 app/main.py, making sure you are in the root repo directory.
Database was created by our own by scraping the LEGO pictures from websites such as gettyimages.com, istockphoto.com and brickset.com (scripts available in the Preprocessing directory). LEGO database with mixed figures and constructions images is available here.
Image captioning was done using BLIP model. Image captioning script (image_captioning.py) is available in the Preprocessing/BLIP project directory.
App is created using gradio library. In order to change GPU index or models' names do it in the app/main.py file.
Models are available here: https://www.kaggle.com/datasets/michasakowicz/lego-dataset-models.
Results
Images:
Rubber Duck Aliens visiting
the Earth for the first time
A small cabin on top of a snowy
mountain in the style of Disney
A high tech solarpunk utopia
in the Amazon rainforest
An insect robot preparing
a delicious meal
| Rubber Duck Aliens visiting the Earth for the first time |
A small cabin on top of a snowy mountain in the style of Disney |
A high tech solarpunk utopia in the Amazon rainforest |
An insect robot preparing a delicious meal |
|---|---|---|---|
 |
 |
 |
 |
App:
