ReFT: Fine Tuned models by AI Generated Images

ReFT is a series of fine-tuned models from SD 1.5 for producing illustration style images with high resolution or different aspect ratio images.

Model Description

Model Name	Base Model	Typical Resolusion	lr
ReFT ReWaifu 512p	SD1.5	512x512	4e-6
ReFT ReWaifu 768p	ReWaifu_512p_e12	768x768	4e-6
ReFT ReWaifu 1K	ReWaifu_768p_e5	1024x1024	2e-6
ReFT Rellust 512p	SD1.5	512x512	4e-6
ReFT Rellust 768p	Rellust_512p_e10	768x768	4e-6
ReFT Rellust 1K	Rellust_768p_e3	1024x1024	4e-6

CLIP Skip 1 is recommended for all models!!

ReFT ReWaifu

Each model is fine-tuned from SD1.5 with AI-Illustration tagged images of various authors published on the web and NijiJourney-Prompt-Pairs using Aspect Ratio Bucketing based on either 512p, 768p, or 1K resolution. For 512p, a 370k image data set was used, and for 768p and 1K a subset of 20k images selected by ImageReward score with BLIP was used. Image captions are made by BLIP and WD1.4-tagger. The batch size is 16 for 512p only and 8 for the rest. Fine tuning is also performed at fp16, using mutires noise.

ReFT Rellust

Each model is fine-tuned from SD1.5 with ~16k nijijourneyv5 tagged images of various authors published on the web with a learning rate of 4e-6 using Aspect Ratio Bucketing based on either 512p, 768p, or 1K resolution. Image captions are made by BLIP and WD1.4-tagger. The batch size is 16 for 512p only and 8 for the rest. Fine tuning is also performed at fp16, using mutires noise.

Usage

Since these models are generic illustration models, the generated images can be in a variety of styles. If the style is not to your liking, please specify a style such as "anime" or "wator colors" as appropriate.

The format of the caption suggests that a short natural language sentence followed by a comma-separated tags is the most natural way to describe the prompt. Using more tags that are well-estimated by the tagger in the trained images may lead to more preferable generation. Tag semantics may be inappropriate for automatic tagging, so please emphasize appropriately. The models do not require underscores for tags in the prompts.

CLIP Skip 1 is recommended.

Examples

ReFT ReWafu 1K

v2 (Improved quality degradation with short prompts, as well as little better detail. If multiple persons appear unintentionally or the anatomy is bad, the first (t2i) resolution should be reduced slightly.)

solo, 1girl, school uniform, white shirt, sailor collar, tie, brown hair, bob cut, scenery, outdoors, solo
Negative prompt: bad anatomy, text, frame
Steps: 30, Sampler: UniPC, CFG scale: 6, Seed: 276042060, Size: 1280x720, Model hash: 5b383d445c, Model: ReWaifu_v2_1K_e4, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1

wide shot, grapes, muscat, white background, simple background, full body, solo, 1girl, small head, light-brown hair, medium hair, wavy hair, brown eyes, simple dress, casual, slim, shoes, water colors, pastel colors, light coloring
Negative prompt: worst quality, low quality, lowres, blurry, text, frame, light particle, water, vivid color, black, messy hair, frills
Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 3916112453, Size: 1024x1024, Model hash: 5b383d445c, Model: ReWaifu_v2_1K_e4, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1

wide shot, solo, 1girl, full body, black hair, twintails, red eyes, white blouse, Peter Pan collar, dress, pinafore dress, strawberry print skirt, waist bow, frilled dress, pink skirt, long sleeves, puffy long sleeves, food print, pink footwear, strap shoes, closed mouth, fruit, white background, strawberry, pastel
Negative prompt: bad anatomy, text, frame, blurry, realistic
Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 2310821, Size: 1024x1024, Model hash: 7716dbfa7b, Model: ReWaifu_1K_e5, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1

solo, 1girl, cowboy shot, magical girl, white hair, medium hair, hair ornament, blue eyes, black dress, frilled dress, short sleeves, puffy sleeves, holding staff of ruby, starry sky, milky way
Negative prompt: bad anatomy, text, frame, blurry, realistic, skirt hold
Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 4266538265, Size: 960x1152, Model hash: 7716dbfa7b, Model: SDAXa3c, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: FWR_TEfixed2(539136a8cf23), AddNet Weight A 1: 0.5, AddNet Weight B 1: 0.5, Version: v1.5.1

wide shot, grapes, muscat, white background, simple background, full body, solo, 1girl, small head, light-brown hair, medium hair, wavy hair, brown eyes, simple dress, casual, slim, shoes, water colors, pastel colors, light coloring
Negative prompt: worst quality, low quality, lowres, blurry, text, frame, light particle, water, vivid color, black, messy hair, frills
Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 2887952460, Size: 1024x1024, Model hash: 7716dbfa7b, Model: ReWaifu_1K_e5, Variation seed: 1970649291, Variation seed strength: 0.1, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1

2d <-> 3d: Use "realistic" tag.
2d

solo, 1girl, from side, anime girl is holding a maple leaf, red hair, own hands together , medium hair, wavy hair, yellow eyes, white sweater, long sleeves, skirt, mini skirt, plaid skirt, strap shoes, out doors, maple hair ornament, scenery, depth of field, tree, maple, look into hands
Negative prompt: bad anatomy, text, frame, blurry, school uniform, realistic, long neck
Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 1783332202, Size: 1024x1024, Model hash: 7716dbfa7b, Model: ReWaifu_1K_e5, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: FWR_TEfixed2(539136a8cf23), AddNet Weight A 1: 0.5, AddNet Weight B 1: 0.5, Version: v1.5.1

solo, 1girl, from side, anime girl is holding a maple leaf, red hair, own hands together , medium hair, wavy hair, yellow eyes, white sweater, long sleeves, skirt, mini skirt, plaid skirt, strap shoes, out doors, maple hair ornament, scenery, depth of field, tree, maple, look into hands, realistic
Negative prompt: bad anatomy, text, frame, blurry, school uniform
Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 1783332202, Size: 1024x1024, Model hash: 7716dbfa7b, Model: ReWaifu_1K_e5, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: FWR_TEfixed2(539136a8cf23), AddNet Weight A 1: 0.5, AddNet Weight B 1: 0.5, Version: v1.5.1

ReFT Rellust 1K

solo, 1girl, full body, white background, indoors, from side, short hair, bangs, long sleeves, closed mouth, white hair, jewelry, blush, standing, earrings dress, flower, shirt, blue eyes, hair ornament, skirt, bow
Negative prompt: worst quality, low quality, bad anatomy blurry, text
Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 4203630207, Size: 1280x720, Model hash: 7d610c9f1e, Model: Rellust_1K_e3, Denoising strength: 0.55, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1

wide shot, cosmos, white background, simple background, full body, solo, 1girl, small head, light-brown hair, medium hair, wavy hair, brown eyes, simple dress, casual, slim, strap shoes, cosmos, water colors, pastel colors, light coloring
Negative prompt: worst quality, low quality, logo, blurry, text, frame, light particle, water, vivid color, black, messy hair, frills
Steps: 30, Sampler: UniPC, CFG scale: 6, Seed: 122211893, Size: 1024x1024, Model hash: 7d610c9f1e, Model: Rellust_1K_e3, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1

wide shot, grapes, white background, simple background, full body, solo, 1girl, small head, light-brown hair, medium hair, wavy hair, brown eyes, simple dress, casual, slim, shoes, muscat, water colors, pastel colors, light coloring, grapes
Negative prompt: worst quality, low quality, lowres, blurry, text, frame, light particle, water, vivid color, black, messy hair, frills
Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 3764144276, Size: 1024x1024, Model hash: 7d610c9f1e, Model: Rellust_1K_e3, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1

License

This model is open access and available to all, with a CreativeML OpenRAIL-M license further specifying rights and usage. The CreativeML OpenRAIL License specifies:

You can't use the model to deliberately produce nor share illegal or harmful outputs or content
The authors claims no rights on the outputs you generate, you are free to use them and are accountable for their use which must not go against the provisions set in the license
You may re-distribute the weights and use the model commercially and/or as a service. If you do, please be aware you have to include the same use restrictions as the ones in the license and share a copy of the CreativeML OpenRAIL-M to all your users (please read the license entirely and carefully) Please read the full license here

Acknowledgements

These Models build on the excellent works: SD1.5, developed by CompVis and RUNWAY, and ImageReward (Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong).