Segmind-VegaRT - Latent Consistency Model (LCM) LoRA of Segmind-Vega
Try real-time inference here VegaRT demoโก
API for Segmind-VegaRT
Segmind-VegaRT a distilled consistency adapter for Segmind-Vega that allows to reduce the number of inference steps to only between 2 - 8 steps.
Latent Consistency Model (LCM) LoRA was proposed in LCM-LoRA: A universal Stable-Diffusion Acceleration Module by Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu et al.
Image comparison (Segmind-VegaRT vs SDXL-Turbo)
Speed comparison (Segmind-VegaRT vs SDXL-Turbo) on A100 80GB
Model | Params / M |
---|---|
lcm-lora-sdv1-5 | 67.5 |
Segmind-VegaRT | 119 |
lcm-lora-sdxl | 197 |
Usage
LCM-LoRA is supported in ๐ค Hugging Face Diffusers library from version v0.23.0 onwards. To run the model, first
install the latest version of the Diffusers library as well as peft
, accelerate
and transformers
.
audio dataset from the Hugging Face Hub:
pip install --upgrade pip
pip install --upgrade diffusers transformers accelerate peft
Text-to-Image
Let's load the base model segmind/Segmind-Vega
first. Next, the scheduler needs to be changed to LCMScheduler
and we can reduce the number of inference steps to just 2 to 8 steps.
Please make sure to either disable guidance_scale
or use values between 1.0 and 2.0.
import torch
from diffusers import LCMScheduler, AutoPipelineForText2Image
model_id = "segmind/Segmind-Vega"
adapter_id = "segmind/Segmind-VegaRT"
pipe = AutoPipelineForText2Image.from_pretrained(model_id, torch_dtype=torch.float16, variant="fp16")
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)
pipe.to("cuda")
# load and fuse lcm lora
pipe.load_lora_weights(adapter_id)
pipe.fuse_lora()
prompt = "Self-portrait oil painting, a beautiful cyborg with golden hair, 8k"
# disable guidance_scale by passing 0
image = pipe(prompt=prompt, num_inference_steps=4, guidance_scale=0).images[0]
- Downloads last month
- 534
Model tree for segmind/Segmind-VegaRT
Base model
segmind/Segmind-Vega