|
--- |
|
license: cc-by-4.0 |
|
datasets: |
|
- calm-and-collected/wish-you-were-here |
|
language: |
|
- en |
|
pipeline_tag: text-to-image |
|
tags: |
|
- art |
|
- vintage |
|
- postcard |
|
- lora |
|
- diffuser |
|
library_name: diffusers |
|
base_model: runwayml/stable-diffusion-v1-5 |
|
instance_prompt: null |
|
widget: |
|
- text: >- |
|
WYWH postcard drawing of a man sitting in a field looking at a mountain |
|
range damage |
|
--- |
|
# Wish You Were Here - a Stable diffusion 1.5 LORA for vintage postcard replication |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
 |
|
|
|
Wish you were here is a LORA model developped to create vintage postcard images. The model was trained on Stable Diffusion 1.5. |
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
Wish You Were Here (WYWH) is a LORA model developped to replicate the look and feel of vintage postcards. This is done via harvesting public domain images from WikiMedia via |
|
manual review and using a combination of manual and automated annotation to describe the images. The specific feature desired to extract were: color, damage and printing |
|
technique. The model was developped over a duration of 2 days over 100 epochs of which one epoch was taken as resulting image. |
|
|
|
|
|
- **Developed by:** calm-and-collected |
|
- **Model type:** LORA |
|
- **License:** CC-BY 4.0 |
|
- **Finetuned from model [optional]:** Stable diffusion 1.5 pruned |
|
|
|
## Bias, Risks, and Limitations |
|
|
|
The model is trained of images from ~650 images. From observation, the majority of these images are from american origins. The model is thus excelent at replicating USA |
|
destinations. The model will also replicate damage seen in the images. |
|
|
|
|
|
### Recommendations |
|
|
|
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> |
|
|
|
To use the WYWH model, use your favorite Stable Diffusion model (the recommended model is a realistic model) and use the LORA along with the following triggers: |
|
- WYWH (the base trigger) |
|
- Photograph (for photography postcards) |
|
- Drawing (for drawn postcards) |
|
- Damage (to add scratch and water damage to the generation) |
|
- Monochrome (for black and white images) |
|
|
|
For negatives, your can use the following: |
|
- White border (if you do not want a white border) |
|
|
|
## How to Get Started with the Model |
|
|
|
You can use this model with [automatic1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui), [comfyui](https://github.com/comfyanonymous/ComfyUI) and [sdnext](https://github.com/vladmandic/automatic). |
|
|
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
The Wish You Were Here dataset consists out of ~650 images of postcards from 1900-1970. |
|
Dataset: [origional dataset](https://huggingface.co/datasets/calm-and-collected/wish_you_were_here "The wish you were here dataset"). |
|
|
|
### Training Hyperparameters |
|
|
|
<details> |
|
<summary>Kohya_SS paramaters</summary> |
|
```js |
|
{ |
|
"LoRA_type": "Standard", |
|
"adaptive_noise_scale": 0, |
|
"additional_parameters": "", |
|
"block_alphas": "", |
|
"block_dims": "", |
|
"block_lr_zero_threshold": "", |
|
"bucket_no_upscale": true, |
|
"bucket_reso_steps": 64, |
|
"cache_latents": true, |
|
"cache_latents_to_disk": true, |
|
"caption_dropout_every_n_epochs": 0.0, |
|
"caption_dropout_rate": 0, |
|
"caption_extension": ".txt", |
|
"clip_skip": 2, |
|
"color_aug": false, |
|
"conv_alpha": 1, |
|
"conv_block_alphas": "", |
|
"conv_block_dims": "", |
|
"conv_dim": 1, |
|
"decompose_both": false, |
|
"dim_from_weights": false, |
|
"down_lr_weight": "", |
|
"enable_bucket": true, |
|
"epoch": 1, |
|
"factor": -1, |
|
"flip_aug": false, |
|
"full_bf16": false, |
|
"full_fp16": false, |
|
"gradient_accumulation_steps": 1, |
|
"gradient_checkpointing": false, |
|
"keep_tokens": "0", |
|
"learning_rate": 0.0001, |
|
"logging_dir": "/home/glow/Desktop/ml/whyw_logs", |
|
"lora_network_weights": "", |
|
"lr_scheduler": "constant", |
|
"lr_scheduler_args": "", |
|
"lr_scheduler_num_cycles": "", |
|
"lr_scheduler_power": "", |
|
"lr_warmup": 0, |
|
"max_bucket_reso": 2048, |
|
"max_data_loader_n_workers": "1", |
|
"max_resolution": "512,650", |
|
"max_timestep": 1000, |
|
"max_token_length": "75", |
|
"max_train_epochs": "100", |
|
"max_train_steps": "", |
|
"mem_eff_attn": true, |
|
"mid_lr_weight": "", |
|
"min_bucket_reso": 256, |
|
"min_snr_gamma": 0, |
|
"min_timestep": 0, |
|
"mixed_precision": "bf16", |
|
"model_list": "custom", |
|
"module_dropout": 0.2, |
|
"multires_noise_discount": 0.2, |
|
"multires_noise_iterations": 8, |
|
"network_alpha": 128, |
|
"network_dim": 256, |
|
"network_dropout": 0.3, |
|
"no_token_padding": false, |
|
"noise_offset": "0.05", |
|
"noise_offset_type": "Multires", |
|
"num_cpu_threads_per_process": 2, |
|
"optimizer": "AdamW8bit", |
|
"optimizer_args": "", |
|
"output_dir": "/home/glow/Desktop/ml/whyw_logs/model_v2", |
|
"output_name": "final_model", |
|
"persistent_data_loader_workers": false, |
|
"pretrained_model_name_or_path": "runwayml/stable-diffusion-v1-5", |
|
"prior_loss_weight": 1.0, |
|
"random_crop": false, |
|
"rank_dropout": 0.2, |
|
"reg_data_dir": "", |
|
"resume": "", |
|
"sample_every_n_epochs": 0, |
|
"sample_every_n_steps": 0, |
|
"sample_prompts": "", |
|
"sample_sampler": "euler_a", |
|
"save_every_n_epochs": 1, |
|
"save_every_n_steps": 0, |
|
"save_last_n_steps": 0, |
|
"save_last_n_steps_state": 0, |
|
"save_model_as": "safetensors", |
|
"save_precision": "bf16", |
|
"save_state": false, |
|
"scale_v_pred_loss_like_noise_pred": false, |
|
"scale_weight_norms": 1, |
|
"sdxl": false, |
|
"sdxl_cache_text_encoder_outputs": false, |
|
"sdxl_no_half_vae": true, |
|
"seed": "1234", |
|
"shuffle_caption": false, |
|
"stop_text_encoder_training": 1, |
|
"text_encoder_lr": 5e-05, |
|
"train_batch_size": 3, |
|
"train_data_dir": "/home/glow/Desktop/wyhw", |
|
"train_on_input": true, |
|
"training_comment": "", |
|
"unet_lr": 0.0001, |
|
"unit": 1, |
|
"up_lr_weight": "", |
|
"use_cp": true, |
|
"use_wandb": false, |
|
"v2": false, |
|
"v_parameterization": false, |
|
"v_pred_like_loss": 0, |
|
"vae_batch_size": 0, |
|
"wandb_api_key": "", |
|
"weighted_captions": false, |
|
"xformers": "xformers" |
|
} |
|
``` |
|
The final model is the 50th iteration of the model. |
|
</details> |
|
|
|
#### Hardware |
|
|
|
The model was trained on two GTX 4090 for a duration of 2 days to extract 100 epochs of the model. |
|
|
|
#### Software |
|
|
|
The model was trained via the Kohya_SS gui. |
|
|
|
## Model Card Contact |
|
|
|
Use the community section of this repository to contact me. |
