thegenerativegeneration/ukiyoe-diffusion-256

Ukiyo-e Diffusion

If you make something using these models, you're welcome to mention me @thegenerativegeneration

Named by dataset used. Current and best version is models/ukiyoe-all/v1/ema_0.9999_056000.pt

Current Plans

• clean dataset
  • remove borders
  • remove some of the samples with text in them

Models

Ukiyo-e-all

v1

models/ukiyoe-all/v1/ema_0.9999_056000.pt

Model configuration is:

model_config = {
    'attention_resolutions': '32, 16, 8',
    'class_cond': False,
    'image_size': 256,
    'learn_sigma': True,
    'rescale_timesteps': True,
    'noise_schedule': 'linear',
    'num_channels': 128,
    'num_heads': 4,
    'num_res_blocks': 2,
    'resblock_updown': True,
    'use_checkpoint': True,
    'use_fp16': True,
    'use_scale_shift_norm': True,
}

Tips

• Results closest to original training data are achieved by turning off the secondary model in Disco Diffusion.
• Turning secondary model on can lead to very creative results
• It is not necessary to specify Ukiyo-e as artstyle to get ukiyo-e-like images.

Examples

If you make something nice using these models, I would like to link your image.

Secondary Off

Secondary On

About

Trained from scratch on a ~170000 images corpus of ukiyo-e.org filtered by colorfulness >= 5.

(Deprecated) Ukiyo-e-few

models/ukiyoe-few/v1/ukiyoe_diffusion_256_022000.pt

Finetuned on 5224 images from Wikiart (1168) and ? ().

Model configuration is

model_config = {
    'attention_resolutions': '16',
    'class_cond': False,
    'diffusion_steps': 1000,
    'rescale_timesteps': True,
    'timestep_respacing': 'ddim100',
    'image_size': 256,
    'learn_sigma': True,
    'noise_schedule': 'linear',
    'num_channels': 128,
    'num_heads': 1,
    'num_res_blocks': 2,
    'use_checkpoint': True,
    'use_scale_shift_norm': False
}

Trained using a fork of guided-diffusion-sxela. Added random crop which did not lead to good results.