File size: 6,331 Bytes
b74d412 cf6235a 453ff2b cf6235a a998317 96b59ab 2acd09a 453ff2b b74d412 958fab0 cf6235a 958fab0 cf6235a 958fab0 cf6235a 958fab0 cf6235a 958fab0 9069a73 958fab0 cf6235a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 |
---
license: cc-by-4.0
datasets:
- calm-and-collected/wish-you-were-here
language:
- en
pipeline_tag: text-to-image
tags:
- art
- vintage
- postcard
- lora
- diffuser
library_name: diffusers
base_model: runwayml/stable-diffusion-v1-5
instance_prompt: null
widget:
- text: >-
WYWH postcard drawing of a man sitting in a field looking at a mountain
range damage
---
# Wish You Were Here - a Stable diffusion 1.5 LORA for vintage postcard replication
<!-- Provide a quick summary of what the model is/does. -->
Wish you were here is a LORA model developped to create vintage postcard images. The model was trained on Stable Diffusion 1.5.
### Model Description
<!-- Provide a longer summary of what this model is. -->
Wish You Were Here (WYWH) is a LORA model developped to replicate the look and feel of vintage postcards. This is done via harvesting public domain images from WikiMedia via
manual review and using a combination of manual and automated annotation to describe the images. The specific feature desired to extract were: color, damage and printing
technique. The model was developped over a duration of 2 days over 100 epochs of which one epoch was taken as resulting image.
- **Developed by:** calm-and-collected
- **Model type:** LORA
- **License:** CC-BY 4.0
- **Finetuned from model [optional]:** Stable diffusion 1.5 pruned
## Bias, Risks, and Limitations
The model is trained of images from ~650 images. From observation, the majority of these images are from american origins. The model is thus excelent at replicating USA
destinations. The model will also replicate damage seen in the images.
### Recommendations
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
To use the WYWH model, use your favorite Stable Diffusion model (the recommended model is a realistic model) and use the LORA along with the following triggers:
- WYWH (the base trigger)
- Photograph (for photography postcards)
- Drawing (for drawn postcards)
- Damage (to add scratch and water damage to the generation)
- Monochrome (for black and white images)
For negatives, your can use the following:
- White border (if you do not want a white border)
## How to Get Started with the Model
You can use this model with [automatic1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui), [comfyui](https://github.com/comfyanonymous/ComfyUI) and [sdnext](https://github.com/vladmandic/automatic).
## Training Details
### Training Data
The Wish You Were Here dataset consists out of ~650 images of postcards from 1900-1970.
Dataset: [origional dataset](https://huggingface.co/datasets/calm-and-collected/wish_you_were_here "The wish you were here dataset").
### Training Hyperparameters
<details>
<summary>Kohya_SS paramaters</summary>
```js
{
"LoRA_type": "Standard",
"adaptive_noise_scale": 0,
"additional_parameters": "",
"block_alphas": "",
"block_dims": "",
"block_lr_zero_threshold": "",
"bucket_no_upscale": true,
"bucket_reso_steps": 64,
"cache_latents": true,
"cache_latents_to_disk": true,
"caption_dropout_every_n_epochs": 0.0,
"caption_dropout_rate": 0,
"caption_extension": ".txt",
"clip_skip": 2,
"color_aug": false,
"conv_alpha": 1,
"conv_block_alphas": "",
"conv_block_dims": "",
"conv_dim": 1,
"decompose_both": false,
"dim_from_weights": false,
"down_lr_weight": "",
"enable_bucket": true,
"epoch": 1,
"factor": -1,
"flip_aug": false,
"full_bf16": false,
"full_fp16": false,
"gradient_accumulation_steps": 1,
"gradient_checkpointing": false,
"keep_tokens": "0",
"learning_rate": 0.0001,
"logging_dir": "/home/glow/Desktop/ml/whyw_logs",
"lora_network_weights": "",
"lr_scheduler": "constant",
"lr_scheduler_args": "",
"lr_scheduler_num_cycles": "",
"lr_scheduler_power": "",
"lr_warmup": 0,
"max_bucket_reso": 2048,
"max_data_loader_n_workers": "1",
"max_resolution": "512,650",
"max_timestep": 1000,
"max_token_length": "75",
"max_train_epochs": "100",
"max_train_steps": "",
"mem_eff_attn": true,
"mid_lr_weight": "",
"min_bucket_reso": 256,
"min_snr_gamma": 0,
"min_timestep": 0,
"mixed_precision": "bf16",
"model_list": "custom",
"module_dropout": 0.2,
"multires_noise_discount": 0.2,
"multires_noise_iterations": 8,
"network_alpha": 128,
"network_dim": 256,
"network_dropout": 0.3,
"no_token_padding": false,
"noise_offset": "0.05",
"noise_offset_type": "Multires",
"num_cpu_threads_per_process": 2,
"optimizer": "AdamW8bit",
"optimizer_args": "",
"output_dir": "/home/glow/Desktop/ml/whyw_logs/model_v2",
"output_name": "final_model",
"persistent_data_loader_workers": false,
"pretrained_model_name_or_path": "runwayml/stable-diffusion-v1-5",
"prior_loss_weight": 1.0,
"random_crop": false,
"rank_dropout": 0.2,
"reg_data_dir": "",
"resume": "",
"sample_every_n_epochs": 0,
"sample_every_n_steps": 0,
"sample_prompts": "",
"sample_sampler": "euler_a",
"save_every_n_epochs": 1,
"save_every_n_steps": 0,
"save_last_n_steps": 0,
"save_last_n_steps_state": 0,
"save_model_as": "safetensors",
"save_precision": "bf16",
"save_state": false,
"scale_v_pred_loss_like_noise_pred": false,
"scale_weight_norms": 1,
"sdxl": false,
"sdxl_cache_text_encoder_outputs": false,
"sdxl_no_half_vae": true,
"seed": "1234",
"shuffle_caption": false,
"stop_text_encoder_training": 1,
"text_encoder_lr": 5e-05,
"train_batch_size": 3,
"train_data_dir": "/home/glow/Desktop/wyhw",
"train_on_input": true,
"training_comment": "",
"unet_lr": 0.0001,
"unit": 1,
"up_lr_weight": "",
"use_cp": true,
"use_wandb": false,
"v2": false,
"v_parameterization": false,
"v_pred_like_loss": 0,
"vae_batch_size": 0,
"wandb_api_key": "",
"weighted_captions": false,
"xformers": "xformers"
}
```
The final model is the 50th iteration of the model.
</details>
#### Hardware
The model was trained on two GTX 4090 for a duration of 2 days to extract 100 epochs of the model.
#### Software
The model was trained via the Kohya_SS gui.
## Model Card Contact
Use the community section of this repository to contact me. |