AlekseyCalvin commited on
Commit
c246c7a
1 Parent(s): a311703

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +128 -93
README.md CHANGED
@@ -5,101 +5,135 @@ tags:
5
  - lora
6
  - diffusers
7
  - template:sd-lora
 
 
8
  - ai-toolkit
9
  widget:
10
- - text: /@step 4000 weights:/ HST style autochrome photo of a drugged God Christ-insect in wooden pillbox
11
- being pinned needle-poked by a giant Marina Tsvetaeva in a long
12
- red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov &
13
- David Lynch styles, detailed face worrying, full-height red-robed woman roams
14
- European city circa 1930
15
- output:
16
- url: samples/1729711624663__000004000_4.jpg
17
- - text: /@step 1600 weights:/HST style autochrome photo of a drugged God Christ-insect in wooden pillbox
18
- being pinned needle-poked by a giant Marina Tsvetaeva in a long
19
- red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov &
20
- David Lynch styles, detailed face worrying, full-height red-robed woman roams
21
- European city circa 1930
22
- output:
23
- url: samples/1729700659095__000001600_4.jpg
24
- - text: /@step 1800 weights:/HST style autochrome photo of a drugged God Christ-insect in wooden pillbox
25
- being pinned needle-poked by a giant Marina Tsvetaeva in a long
26
- red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov &
27
- David Lynch styles, detailed face worrying, full-height red-robed woman roams
28
- European city circa 1930
29
- output:
30
- url: samples/1729701571749__000001800_4.jpg
31
- - text: /@step 800 weights:/HHST style autochrome photo of a drugged God Christ-insect in wooden pillbox
32
- being pinned needle-poked by a giant Marina Tsvetaeva in a long
33
- red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov &
34
- David Lynch styles, detailed face worrying, full-height red-robed woman roams
35
- European city circa 1930
36
- output:
37
- url: samples/1729696985671__000000800_4.jpg
38
- - text: '/@step 1800 weights:/HST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich,
39
- dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing
40
- with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh. '
 
 
 
 
 
 
 
 
41
  output:
42
  url: samples/1729701488064__000001800_0.jpg
43
- - text: '/@step 4000 weights:/HHST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich,
44
- dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing
45
- with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh. '
 
 
 
 
46
  output:
47
  url: samples/1729711540947__000004000_0.jpg
48
- - text: '/@step 1600 weights:/ HST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich,
49
- dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing
50
- with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh. '
 
 
 
 
51
  output:
52
  url: samples/1729700575417__000001600_0.jpg
53
- - text: '/@step 1600 weights:/ HST autochrome style analog dslr award-winning 8k art photo showing a nurse
54
- battling a giant cell phone, above text caption of \ONCE WE WERE ALL HARRY
55
- GARDNER!\, in a highly realistic modern American medical hospital '
56
- output:
57
- url: samples/1729700596338__000001600_1.jpg
58
- - text: '/@step 1000 weights:/ HST autochrome style analog dslr award-winning 8k art photo showing a nurse
59
- battling a giant cell phone, above text caption of \ONCE WE WERE ALL HARRY
60
- GARDNER!\, in a highly realistic modern American medical hospital '
61
- output:
62
- url: samples/1729697845170__000001000_1.jpg
63
- - text: /@step 3600 weights:/ HST style autochrome photograph of a dark CIA agent Koala leaping at the
64
- throat of goth freedom-fighter Julian Assange, tragic atmosphere, award-winning
65
- art photo
66
- output:
67
- url: samples/1729709764758__000003600_2.jpg
68
- - text: /@step 1200 weights:/ HST style autochrome photograph of a dark CIA agent Koala leaping at the
69
- throat of goth freedom-fighter Julian Assange, tragic atmosphere, award-winning
70
- art photo
71
- output:
72
- url: samples/1729698783304__000001200_2.jpg
73
- - text: /@step 1000 weights:/ HST style autochrome photograph of a dark CIA agent Koala leaping at the
74
- throat of goth freedom-fighter Julian Assange, tragic atmosphere, award-winning
75
- art photo
76
- output:
77
- url: samples/1729697866093__000001000_2.jpg
78
- - text: /@step 4000 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent
79
- regions of white fur, playing a piano and singing, amateur 2004 photograph
80
- shot on a cell phone in a Los Angeles apartment kitchen
81
- output:
82
- url: samples/1729711603742__000004000_3.jpg
83
- - text: /@step 3000 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent
84
- regions of white fur, playing a piano and singing, amateur 2004 photograph
85
- shot on a cell phone in a Los Angeles apartment kitchen
86
- output:
87
- url: samples/1729707029633__000003000_3.jpg
88
- - text: /@step 1800 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent
89
- regions of white fur, playing a piano and singing, amateur 2004 photograph
90
- shot on a cell phone in a Los Angeles apartment kitchen
91
- output:
92
- url: samples/1729701550832__000001800_3.jpg
93
- - text: /@step 1000 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent
94
- regions of white fur, playing a piano and singing, amateur 2004 photograph
95
- shot on a cell phone in a Los Angeles apartment kitchen
96
- output:
97
- url: samples/1729697887015__000001000_3.jpg
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
98
  base_model: stabilityai/stable-diffusion-3.5-large
99
-
100
  license: creativeml-openrail-m
101
-
102
-
 
 
103
  ---
104
 
105
  # HSTsd3ii
@@ -110,12 +144,14 @@ Model trained with [AI Toolkit by Ostris](https://github.com/ostris/ai-toolkit)
110
 
111
  'HST style autochrome photo'
112
 
113
- ## Config Parameters
114
- Using the Google Colab Notebook Version of ai-toolkit.
 
115
  I've used A100 via Colab Pro.
116
- But training SD3.5 may potentially work with Free Colab or lower Vram in general, especially if one used lower rank (try 4 or 8), dataset size (in terms of caching/bucketing/pre-loading impacts), 1 batch size, Adamw8bit optimizer, 512 resolution, maybe adding the "lowvram, true" argument, and plausibly specifying alternate quantization variants!
117
- Generally, VRAM expenditures tend to be lower than for Flux during training. So, try it! I certainly will.
118
- To use on Colab, modify a Flux template Notebook from [here](https://github.com/ostris/ai-toolkit/tree/main/notebooks) with parameters from Ostris' example config for SD3.5 [here](https://github.com/ostris/ai-toolkit/blob/main/config/examples/train_lora_sd35_large_24gb.yaml)!
 
119
  ```
120
  job: extension
121
  config:
@@ -165,7 +201,7 @@ config:
165
  is_v3: true
166
  quantize: true
167
  ```
168
- Plus validation settings: Prompts like the above, at 1024, guidance scale 4, 25 steps, seed 42, no negatives.
169
 
170
  ## Download model and use it with ComfyUI, AUTOMATIC1111, SD.Next, Invoke AI, etc.
171
 
@@ -185,5 +221,4 @@ image = pipeline('HST style communist poster with text "JOIN RCA!", over autochr
185
  image.save("my_image.png")
186
  ```
187
 
188
- For more details, including weighting, merging and fusing LoRAs, check the [documentation on loading LoRAs in diffusers](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters)
189
-
 
5
  - lora
6
  - diffusers
7
  - template:sd-lora
8
+ - StableDiffusion3
9
+ - StableDiffusion3.5
10
  - ai-toolkit
11
  widget:
12
+ - text: >-
13
+ /@step 4000 weights:/ HST style autochrome photo of a drugged God
14
+ Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina
15
+ Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film
16
+ in Yury Annenkov & David Lynch styles, detailed face worrying, full-height
17
+ red-robed woman roams European city circa 1930
18
+ output:
19
+ url: samples/1729711624663__000004000_4.jpg
20
+ - text: >-
21
+ /@step 1600 weights:/HST style autochrome photo of a drugged God
22
+ Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina
23
+ Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film
24
+ in Yury Annenkov & David Lynch styles, detailed face worrying, full-height
25
+ red-robed woman roams European city circa 1930
26
+ output:
27
+ url: samples/1729700659095__000001600_4.jpg
28
+ - text: >-
29
+ /@step 1800 weights:/HST style autochrome photo of a drugged God
30
+ Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina
31
+ Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film
32
+ in Yury Annenkov & David Lynch styles, detailed face worrying, full-height
33
+ red-robed woman roams European city circa 1930
34
+ output:
35
+ url: samples/1729701571749__000001800_4.jpg
36
+ - text: >-
37
+ /@step 800 weights:/HHST style autochrome photo of a drugged God
38
+ Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina
39
+ Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film
40
+ in Yury Annenkov & David Lynch styles, detailed face worrying, full-height
41
+ red-robed woman roams European city circa 1930
42
+ output:
43
+ url: samples/1729696985671__000000800_4.jpg
44
+ - text: >-
45
+ /@step 1800 weights:/HST style communist poster with text "JOIN RCA!", over
46
+ autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich,
47
+ dancing with red feathered drunken dinosaur, an early conceptual artist.
48
+ Lenin is full of contageous awe, his blemished skin flushing with anxious
49
+ excitement, his famous bald spot sweatily glistening under warm lights. In
50
+ the back, Krupskaya and Inessa Armand laugh.
51
  output:
52
  url: samples/1729701488064__000001800_0.jpg
53
+ - text: >-
54
+ /@step 4000 weights:/HHST style communist poster with text "JOIN RCA!", over
55
+ autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich,
56
+ dancing with red feathered drunken dinosaur, an early conceptual artist.
57
+ Lenin is full of contageous awe, his blemished skin flushing with anxious
58
+ excitement, his famous bald spot sweatily glistening under warm lights. In
59
+ the back, Krupskaya and Inessa Armand laugh.
60
  output:
61
  url: samples/1729711540947__000004000_0.jpg
62
+ - text: >-
63
+ /@step 1600 weights:/ HST style communist poster with text "JOIN RCA!", over
64
+ autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich,
65
+ dancing with red feathered drunken dinosaur, an early conceptual artist.
66
+ Lenin is full of contageous awe, his blemished skin flushing with anxious
67
+ excitement, his famous bald spot sweatily glistening under warm lights. In
68
+ the back, Krupskaya and Inessa Armand laugh.
69
  output:
70
  url: samples/1729700575417__000001600_0.jpg
71
+ - text: >-
72
+ /@step 1600 weights:/ HST autochrome style analog dslr award-winning 8k art
73
+ photo showing a nurse battling a giant cell phone, above text caption of
74
+ \ONCE WE WERE ALL HARRY GARDNER!\, in a highly realistic modern American
75
+ medical hospital
76
+ output:
77
+ url: samples/1729700596338__000001600_1.jpg
78
+ - text: >-
79
+ /@step 1000 weights:/ HST autochrome style analog dslr award-winning 8k art
80
+ photo showing a nurse battling a giant cell phone, above text caption of
81
+ \ONCE WE WERE ALL HARRY GARDNER!\, in a highly realistic modern American
82
+ medical hospital
83
+ output:
84
+ url: samples/1729697845170__000001000_1.jpg
85
+ - text: >-
86
+ /@step 3600 weights:/ HST style autochrome photograph of a dark CIA agent
87
+ Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic
88
+ atmosphere, award-winning art photo
89
+ output:
90
+ url: samples/1729709764758__000003600_2.jpg
91
+ - text: >-
92
+ /@step 1200 weights:/ HST style autochrome photograph of a dark CIA agent
93
+ Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic
94
+ atmosphere, award-winning art photo
95
+ output:
96
+ url: samples/1729698783304__000001200_2.jpg
97
+ - text: >-
98
+ /@step 1000 weights:/ HST style autochrome photograph of a dark CIA agent
99
+ Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic
100
+ atmosphere, award-winning art photo
101
+ output:
102
+ url: samples/1729697866093__000001000_2.jpg
103
+ - text: >-
104
+ /@step 4000 weights:/ HST style autochrome photo of realistic green-eyed
105
+ black cat, with prominent regions of white fur, playing a piano and singing,
106
+ amateur 2004 photograph shot on a cell phone in a Los Angeles apartment
107
+ kitchen
108
+ output:
109
+ url: samples/1729711603742__000004000_3.jpg
110
+ - text: >-
111
+ /@step 3000 weights:/ HST style autochrome photo of realistic green-eyed
112
+ black cat, with prominent regions of white fur, playing a piano and singing,
113
+ amateur 2004 photograph shot on a cell phone in a Los Angeles apartment
114
+ kitchen
115
+ output:
116
+ url: samples/1729707029633__000003000_3.jpg
117
+ - text: >-
118
+ /@step 1800 weights:/ HST style autochrome photo of realistic green-eyed
119
+ black cat, with prominent regions of white fur, playing a piano and singing,
120
+ amateur 2004 photograph shot on a cell phone in a Los Angeles apartment
121
+ kitchen
122
+ output:
123
+ url: samples/1729701550832__000001800_3.jpg
124
+ - text: >-
125
+ /@step 1000 weights:/ HST style autochrome photo of realistic green-eyed
126
+ black cat, with prominent regions of white fur, playing a piano and singing,
127
+ amateur 2004 photograph shot on a cell phone in a Los Angeles apartment
128
+ kitchen
129
+ output:
130
+ url: samples/1729697887015__000001000_3.jpg
131
  base_model: stabilityai/stable-diffusion-3.5-large
 
132
  license: creativeml-openrail-m
133
+ language:
134
+ - en
135
+ pipeline_tag: text-to-image
136
+ library_name: diffusers
137
  ---
138
 
139
  # HSTsd3ii
 
144
 
145
  'HST style autochrome photo'
146
 
147
+ ## Config Parameters
148
+ *Dim:16 Alpha:32 Optimizer:Ademamix8bit LR:2e-4* **More info below!** <br>
149
+ Fine-tuned using the **Google Colab Notebook*** Version of **ai-toolkit**.<br>
150
  I've used A100 via Colab Pro.
151
+ However, training SD3.5 may potentially work with Free Colab or lower Vram in general:<br>
152
+ Especially if one were to use lower rank (try 4 or 8), dataset size (in terms of caching/bucketing/pre-loading impacts), 1 batch size, Adamw8bit optimizer, 512 resolution, maybe adding the "lowvram, true" argument, and plausibly specifying alternate quantization variants!
153
+ Generally, VRAM expenditures tend to be lower than for Flux during training. So, try it! I certainly will.<br>
154
+ **To use on Colab**, modify a Flux template Notebook from [here](https://github.com/ostris/ai-toolkit/tree/main/notebooks) with parameters from Ostris' example config for SD3.5 [here](https://github.com/ostris/ai-toolkit/blob/main/config/examples/train_lora_sd35_large_24gb.yaml)!
155
  ```
156
  job: extension
157
  config:
 
201
  is_v3: true
202
  quantize: true
203
  ```
204
+ Plus validation settings:<br> Prompts like the above, at 1024, guidance scale 4, 25 steps, seed 42, no negatives.
205
 
206
  ## Download model and use it with ComfyUI, AUTOMATIC1111, SD.Next, Invoke AI, etc.
207
 
 
221
  image.save("my_image.png")
222
  ```
223
 
224
+ For more details, including weighting, merging and fusing LoRAs, check the [documentation on loading LoRAs in diffusers](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters)