balgot/text-2-stylegan3

Text-to-StyleGAN3 Latent Space Translation

This model was created as a part of the project for FI:PA228 (Masaryk University), inspired by this paper: Face Generation from Textual Features using Conditionally trained Inputs to Generative Adversarial Networks

It was trained on the generated dataset from BLIP and StyleGAN3. See the corresponding notebook for further details.

How to use:

import torch.nn as nn


# for now, the model class needs to be defined, so...
class LaTran(nn.Module):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.pipe = nn.Sequential(
            nn.Linear(384, 512),
            nn.ReLU(),
            nn.Linear(512, 512)
        )

    def forward(self, v):
        return self.pipe(v.unsqueeze(1))



# Instantiate and load the model
dev = ...  # device to use
PATH = "translation_model-sd.pt"  # local path
model = LaTran().to(dev)
model.load_state_dict(torch.load(TRANSLATION_MODEL, map_location=dev))

Demo

For the demo of the whole pipeline, or how this model helps to generate a final image, visits text-to-stylegan HF space.

Examples

• Prompt: attractive young woman, blond hair

• Prompt initial: cute young boy, blond hair, blue eyes, smiling

• Prompt second: old man, short gray hair, glasses, wearing hat