Update README.md

README.md

@@ -1,3 +1,124 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+datasets:
+- KBlueLeaf/danbooru2023-webp-4Mpixel
+- KBlueLeaf/danbooru2023-metadata-database
+base_model:
+- black-forest-labs/FLUX.1-schnell
+- FA770/Sumeshi_Flux.1_S_v002E
+pipeline_tag: text-to-image
+tags:
+- anime
+- girls
+---
+
+![sample_image](./sample_images/1.webp)
+
+# Model Information
+
+**Note:** This model is a Schnell-based model, but it requires CFG scale of 3 or higher (not guidance scale) with 20 steps or more. It needs to be used with `clip_l_nigemono_flux1C`.
+**At this time, this model cannot be generated in WebUI Forge. Please use it with ComfyUI.**
+
+My English is terrible, so I use translation tools.
+
+## Description
+Nigemono_Flux.1_Compact7.5B is an experimental anime model to verify the reduction of parameters in the Flux model. The number of parameters has been reduced from `12B` (double blocks 19/ single blocks 38) to `7.5B` (double blocks 12/ single blocks 24). You can use a negative prompt which works to some extent. The output is blurred and the style varies depending on the prompt, perhaps because the model has not been fully trained.
+
+## Usage
+- Resolution: Like other Flux models
+- **(Distilled) Guidance Scale:** (Distilled) Guidance Scale: 0 (Does not work due to Schnell-based model)
+- **CFG Scale:** 6 ~ 9 (recommend 7; scale 1 does not generate decent outputs)
+- **Steps:** 20 ~ 100 or and more (recommend 40)
+- sampler: Euler
+- scheduler: Simple, Beta
+
+## Prompt Format (from [Kohaku-XL-Epsilon](https://huggingface.co/KBlueLeaf/Kohaku-XL-Epsilon))
+```<1girl/1boy/1other/...>, <character>, <series>, <artists>, <general tags>, <quality tags>, <year tags>, <meta tags>, <rating tags>```
+
+Due to the small amount of training, the `<character><series><artists>` tags are almost non-functional. As training is focused on girl characters, it may not generate boy or other non-persons well. Since the dataset was created using hakubooru, the prompt format will be the same as the KohakuXL format. However, based on experiments, it is not strictly necessary to follow this format, as it interprets meaning to some extent even in natural language.
+
+### Special Tags
+- **Quality Tags:** masterpiece, best quality, great quality, good quality, normal quality, low quality, worst quality
+- **Rating Tags:** safe, sensitive, nsfw, explicit
+- **Date Tags:** newest, recent, mid, early, old
+
+## Training
+
+### Dataset Preparation
+I used [hakubooru](https://github.com/KohakuBlueleaf/HakuBooru)-based custom scripts.
+
+- **Exclude Tags:** `traditional_media, photo_(medium), scan, animated, animated_gif, lowres, non-web_source, variant_set, tall image, duplicate, pixel-perfect_duplicate`
+- **Minimum Post ID:** 1,000,000
+
+### Blocks Reduction
+I reduced the parameters by gradually removing blocks and retraining, so as not to completely disrupt the model's generation capability. Removing too many blocks at once results in completely noisy output, requiring retraining from scratch.
+
+- **Training Hardware:** A single RTX 4090
+- **Method:** LoRA training and merging the results
+- **Training Script:** [sd-scripts](https://github.com/kohya-ss/sd-scripts)
+- **Basic Settings:**
+```powershell
+accelerate launch --num_cpu_threads_per_process 4 flux_train_network.py --network_module networks.lora_flux --sdpa --gradient_checkpointing --cache_latents --cache_latents_to_disk --cache_text_encoder_outputs  --cache_text_encoder_outputs_to_disk --max_data_loader_n_workers 1 --save_model_as "safetensors" --mixed_precision "bf16" --fp8_base --save_precision "bf16" --full_bf16 --min_bucket_reso 384 --max_bucket_reso 1536 --seed 1 --max_train_epochs 1 --keep_tokens_separator "|||" --network_dim 32 --network_alpha 32 --unet_lr 1e-4 --train_batch_size 2 --gradient_accumulation_steps 3 --optimizer_type adamw8bit --lr_scheduler="constant_with_warmup" --lr_warmup_steps 500 --vae_batch_size 4 --cache_info --guidance_scale 1 --timestep_sampling shift --model_prediction_type raw --discrete_flow_shift 1.8 --loss_type l2 --highvram --network_args "in_dims=[16,16,16,0,16]" --network_train_unet_only --bucket_no_upscale
+```
+
+--Continued training from sumeshi flux.1s v002E--
+
+1. remove double blocks 18 / single blocks 34,35,36,37 / guidance_in (sumeshi flux.1s has zero tensor)
+
+2. 3,893images (res1024 bs2 /res512 bs8 acc2 warmup50 --lr_scheduler="cosine_with_restarts" --lr_scheduler_num_cycles 1 --discrete_flow_shift 1.8) 4epochs
+
+3. merged into model
+
+4. remove double blocks 16,17 / single blocks 32,33
+
+5. 3,893images (res1024 bs2 /res512 bs8 acc1 warmup50 --lr_scheduler="cosine_with_restarts" --lr_scheduler_num_cycles 2 --discrete_flow_shift 1.8) 2epochs
+
+6. merged into model
+  
+7. remove double blocks 14,15 / single blocks 28,29,30,31
+
+8. 3,893images (res1024 bs2 /res512 bs8 acc2 warmup50 --lr_scheduler="cosine_with_restarts" --lr_scheduler_num_cycles 2 --discrete_flow_shift 1.8) 2epochs
+
+9. merged into model
+
+10. remove double blocks 12,13 / single blocks 24,25,26,27
+
+11. 3,893images (res1024 bs2 /res512 bs8 acc2 warmup50 --lr_scheduler="cosine_with_restarts" --lr_scheduler_num_cycles 2 --discrete_flow_shift 1.8) 2epochs
+
+12. merged into model
+
+13. 3,893images (Full-finetuned res1024 bs1 acc1 afafactor --optimizer_args "relative_step=False" "scale_parameter=False" "warmup_init=False" lr5e-6 warmup100 max_grad_norm 0.0 discrete_flow_shift 2) 2epochs
+
+14. 12,000images (res1024 bs2 acc5 warmup100 --discrete_flow_shift 2) 1epochs
+
+15. 12,000images (res1024 bs2 acc3 warmup100 --discrete_flow_shift 2) 2epochs
+
+16. merged into model
+
+17. 12,000images (res1024 bs2 acc3 warmup100 --discrete_flow_shift 2.5) 3epochs
+
+18. 12,000images (res1024 bs2 acc3 warmup100 --discrete_flow_shift 2.5) 3epochs
+
+19. 12,000images (res512 bs8 acc1 warmup100 timestep_sampling flux_shift(fixed ver.) ) 12epochs
+
+20. merged into model
+
+21. 12,000images (res512 bs4 acc1 warmup100 timesteps_sampling sigmoid sigmoid_scale 0.75) 7epochs
+
+22. 3,893images (res512 bs4 acc1 warmup100 timesteps_sampling sigmoid sigmoid_scale 0.6 --caption_dropout_rate 0.1) 16epochs
+
+23. merged into model
+
+24. 3,893images (res1024 bs2 / res512 bs4 acc2 warmup100 unet_lr1e-4 te_lr5e-5 timesteps_sampling sigmoid sigmoid_scale 0.6 --caption_dropout_rate 0.1) 6epochs
+
+25. merged into model and CLIP_L
+
+## Resources (License)
+- **FLUX.1-schnell (Apache2.0)**  
+- **danbooru2023-webp-4Mpixel (MIT)**
+- **danbooru2023-metadata-database (MIT)**
+
+## Acknowledgements
+- **black-forest-labs:** Thanks for publishing a great open source model.
+- **kohya-ss:** Thanks for publishing the essential training scripts and for the quick updates.
+- **Kohaku-Blueleaf:** Thanks for the extensive publication of the scripts for the dataset and the various training conditions.