---
license: other
license_name: flux-1-dev-non-commercial-license
tags:
- text-to-image
- SVDQuant
- FLUX.1-dev
- INT4
- FLUX.1
- Diffusion
- Quantization
- LoRA
language:
- en
base_model:
- mit-han-lab/svdq-int4-flux.1-dev
- XLabs-AI/flux-RealismLora
- aleksa-codes/flux-ghibsky-illustration
- alvdansen/sonny-anime-fixed
- Shakker-Labs/FLUX.1-dev-LoRA-Children-Simple-Sketch
- linoyts/yarn_art_Flux_LoRA
pipeline_tag: text-to-image
datasets:
- mit-han-lab/svdquant-datasets
library_name: diffusers
---

<p align="center" style="border-radius: 10px">
  <img src="https://github.com/mit-han-lab/nunchaku/raw/refs/heads/main/assets/logo.svg" width="50%" alt="logo"/>
</p>
<h4 style="display: flex; justify-content: center; align-items: center; text-align: center;">Quantization Library:&nbsp;<a href='https://github.com/mit-han-lab/deepcompressor'>DeepCompressor</a> &ensp; Inference Engine:&nbsp;<a href='https://github.com/mit-han-lab/nunchaku'>Nunchaku</a>
</h4>
<div style="display: flex; justify-content: center; align-items: center; text-align: center;">
  <a href="https://arxiv.org/abs/2411.05007">[Paper]</a>&ensp;
  <a href='https://github.com/mit-han-lab/nunchaku'>[Code]</a>&ensp;
  <a href='https://hanlab.mit.edu/projects/svdquant'>[Website]</a>&ensp;
  <a href='https://hanlab.mit.edu/blog/svdquant'>[Blog]</a>
</div>

![teaser](https://github.com/mit-han-lab/nunchaku/raw/refs/heads/main/assets/lora.jpg)
SVDQuant seamlessly integrates with off-the-shelf LoRAs without requiring re-quantization. When applying LoRAs, it matches the image quality of the original 16-bit FLUX.1-dev.

## Model Description

<div>
This reposity contains a converted LoRA collection for SVDQuant INT4 FLUX.1-dev. The LoRA style includes <a href="https://huggingface.co/XLabs-AI/flux-RealismLora">Realism</a>, 
<a href="https://huggingface.co/aleksa-codes/flux-ghibsky-illustration">Ghibsky Illustration</a>, 
<a href="https://huggingface.co/alvdansen/sonny-anime-fixed">Anime</a>, 
<a href="https://huggingface.co/Shakker-Labs/FLUX.1-dev-LoRA-Children-Simple-Sketch">Children Sketch</a>, and 
<a href="https://huggingface.co/linoyts/yarn_art_Flux_LoRA">Yarn Art</a>.
</div>


## Usage

### Diffusers

Please follow the instructions in [mit-han-lab/nunchaku](https://github.com/mit-han-lab/nunchaku) to set up the environment. Then you can run the model with

```python
import torch

from nunchaku.pipelines import flux as nunchaku_flux

pipeline = nunchaku_flux.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    torch_dtype=torch.bfloat16,
    qmodel_path="mit-han-lab/svdq-int4-flux.1-dev",  # download from Huggingface
).to("cuda")
pipeline.transformer.nunchaku_update_params(mit-han-lab/svdquant-models/svdq-flux.1-dev-lora-anime.safetensors)
pipeline.transformer.nunchaku_set_lora_scale(1)
image = pipeline("a dog wearing a wizard hat", num_inference_steps=28, guidance_scale=3.5).images[0]
image.save("example.png")
```

### Comfy UI

Work in progress.

## Limitations

- The model is only runnable on NVIDIA GPUs with architectures sm_86 (Ampere: RTX 3090, A6000), sm_89 (Ada: RTX 4090), and sm_80 (A100). See this [issue](https://github.com/mit-han-lab/nunchaku/issues/1) for more details.
- You may observe some slight differences from the BF16 models in details.

### Citation

If you find this model useful or relevant to your research, please cite

```bibtex
@article{
  li2024svdquant,
  title={SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models},
  author={Li*, Muyang and Lin*, Yujun and Zhang*, Zhekai and Cai, Tianle and Li, Xiuyu and Guo, Junxian and Xie, Enze and Meng, Chenlin and Zhu, Jun-Yan and Han, Song},
  journal={arXiv preprint arXiv:2411.05007},
  year={2024}
}
```