Update README.md
Browse files
README.md
CHANGED
@@ -9,12 +9,28 @@ instance_prompt: null
|
|
9 |
license: mit
|
10 |
---
|
11 |
📢 [[Project Page](https://ali-vilab.github.io/In-Context-LoRA-Page/)] [[Github Repo](https://github.com/ali-vilab/In-Context-LoRA)] [[Paper](https://arxiv.org/abs/2410.23775)]
|
|
|
|
|
|
|
|
|
12 |
# Model Summary
|
13 |
|
14 |
In-Context LoRA fine-tunes text-to-image models (*e.g.,* [FLUX](https://huggingface.co/black-forest-labs/FLUX.1-dev)) to generate image sets with customizable intrinsic relationships, optionally conditioned on another set using SDEdit. It can be adapted to a wide range of tasks
|
15 |
|
16 |
This model hub includes In-Context LoRA models across 10 tasks. [MODEL ZOO](#model-zoo) details these models and their recommend settings. For more details on how these models are trained, please refer to our [paper](https://arxiv.org/abs/2410.23775).
|
17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
## MODEL ZOO
|
19 |
|
20 |
Below lists 10 In-Context LoRA models and their recommend settings.
|
@@ -32,13 +48,30 @@ Below lists 10 In-Context LoRA models and their recommend settings.
|
|
32 |
| **9. Sparklers Visual Effect** | [`sparklers-visual-effect.safetensors`](https://huggingface.co/ali-vilab/In-Context-LoRA/blob/main/sparklers-visual-effect.safetensors) | `width: 960, height: 1088` | `[REAL-SPARKLERS-OVERLAYS] The two-part image vividly illustrates a woodland proposal transformed by sparkler overlays; [TOP] the first panel depicts a man kneeling on one knee with an engagement ring before his partner in a forest clearing at dusk, with warm, natural lighting, [BOTTOM] while the second panel introduces glowing sparklers that form a heart shape around the couple, amplifying the romance and joy of the moment.` |
|
33 |
| **10. Visual Identity Design** | [`visual-identity-design.safetensors`](https://huggingface.co/ali-vilab/In-Context-LoRA/blob/main/visual-identity-design.safetensors) | `width: 1472, height: 1024` | `The two-panel image showcases the joyful identity of a produce brand, with the left panel showing a smiling pineapple graphic and the brand name “Fresh Tropic” in a fun, casual font on a light aqua background; [LEFT] while the right panel translates the design onto a reusable shopping tote with the pineapple logo in black, held by a person in a market setting, emphasizing the brand’s approachable and eco-friendly vibe.` |
|
34 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
35 |
## LICENSE
|
36 |
|
37 |
This model hub uses FLUX as the base model. Users must comply with FLUX's license when using this code. Please refer to [FLUX's License](https://github.com/black-forest-labs/flux/tree/main/model_licenses) for more details.
|
38 |
|
39 |
## Citation
|
40 |
|
41 |
-
If you find this
|
42 |
|
43 |
```bibtex
|
44 |
@article{lhhuang2024iclora,
|
@@ -49,6 +82,15 @@ If you find this model hub useful in your research, please consider citing:
|
|
49 |
}
|
50 |
```
|
51 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
52 |
## Download model
|
53 |
|
54 |
Weights for these models are available in Safetensors format.
|
|
|
9 |
license: mit
|
10 |
---
|
11 |
📢 [[Project Page](https://ali-vilab.github.io/In-Context-LoRA-Page/)] [[Github Repo](https://github.com/ali-vilab/In-Context-LoRA)] [[Paper](https://arxiv.org/abs/2410.23775)]
|
12 |
+
# 🔥 Latest News
|
13 |
+
|
14 |
+
- **[2024-11-16]** 🌟 The community continues to innovate with IC-LoRA! Exciting projects include models, ComfyUI nodes and workflows for **Virtual Try-on, Product Design, Object Mitigation, Role Play**, and more. Explore their creations in **[Community Creations Using IC-LoRA](#community-creations-using-ic-lora)**. Huge thanks to all contributors for their incredible efforts!
|
15 |
+
|
16 |
# Model Summary
|
17 |
|
18 |
In-Context LoRA fine-tunes text-to-image models (*e.g.,* [FLUX](https://huggingface.co/black-forest-labs/FLUX.1-dev)) to generate image sets with customizable intrinsic relationships, optionally conditioned on another set using SDEdit. It can be adapted to a wide range of tasks
|
19 |
|
20 |
This model hub includes In-Context LoRA models across 10 tasks. [MODEL ZOO](#model-zoo) details these models and their recommend settings. For more details on how these models are trained, please refer to our [paper](https://arxiv.org/abs/2410.23775).
|
21 |
|
22 |
+
## Key Idea
|
23 |
+
|
24 |
+
The core concept of IC-LoRA is to **concatenate** both condition and target images into a single composite image while using **Natural Language** to define the task. This approach enables seamless adaptation to a wide range of applications.
|
25 |
+
|
26 |
+
## Features
|
27 |
+
|
28 |
+
- **Task-Agnostic Framework**: IC-LoRA serves as a general framework, but it requires task-specific fine-tuning for diverse applications.
|
29 |
+
- **Customizable Image-Set Generation**: You can fine-tune text-to-image models to **generate image sets** with customizable intrinsic relationships.
|
30 |
+
- **Condition on Image-Set**: You can also **condition the generation of a set of images on another set of images**, enabling a wide range of controllable generation applications.
|
31 |
+
|
32 |
+
For more detailed information and examples, please read our [Paper](https://arxiv.org/abs/2410.23775) or visit our [Project Page](https://ali-vilab.github.io/In-Context-LoRA-Page/).
|
33 |
+
|
34 |
## MODEL ZOO
|
35 |
|
36 |
Below lists 10 In-Context LoRA models and their recommend settings.
|
|
|
48 |
| **9. Sparklers Visual Effect** | [`sparklers-visual-effect.safetensors`](https://huggingface.co/ali-vilab/In-Context-LoRA/blob/main/sparklers-visual-effect.safetensors) | `width: 960, height: 1088` | `[REAL-SPARKLERS-OVERLAYS] The two-part image vividly illustrates a woodland proposal transformed by sparkler overlays; [TOP] the first panel depicts a man kneeling on one knee with an engagement ring before his partner in a forest clearing at dusk, with warm, natural lighting, [BOTTOM] while the second panel introduces glowing sparklers that form a heart shape around the couple, amplifying the romance and joy of the moment.` |
|
49 |
| **10. Visual Identity Design** | [`visual-identity-design.safetensors`](https://huggingface.co/ali-vilab/In-Context-LoRA/blob/main/visual-identity-design.safetensors) | `width: 1472, height: 1024` | `The two-panel image showcases the joyful identity of a produce brand, with the left panel showing a smiling pineapple graphic and the brand name “Fresh Tropic” in a fun, casual font on a light aqua background; [LEFT] while the right panel translates the design onto a reusable shopping tote with the pineapple logo in black, held by a person in a market setting, emphasizing the brand’s approachable and eco-friendly vibe.` |
|
50 |
|
51 |
+
## Community Creations Using IC-LoRA
|
52 |
+
|
53 |
+
We are thrilled to showcase the community's innovative projects leveraging In-Context LoRA (IC-LoRA). If you have additional recommendations or projects to share, **please don't hesitate to send a [Pull Request](https://github.com/ali-vilab/In-Context-LoRA/pulls)!**
|
54 |
+
|
55 |
+
| Project Name | Type | Supported Tasks | Sample Results |
|
56 |
+
|--------------|----------------------|---------------------------------------------------------------------------------|----------------|
|
57 |
+
| 1. [Comfyui_Object_Migration](https://github.com/TTPlanetPig/Comfyui_Object_Migration) | ComfyUI Node & Workflow & LoRA Model | Clothing Migration, Cartoon Clothing to Realism, and More | ![Sample Result](./images/386534865-9612cf8a-858d-4684-819e-7b97981d993c.png) |
|
58 |
+
| 2. [Flux Simple Try On - In Context Lora](https://civitai.com/models/950111/flux-simple-try-on-in-context-lora) | LoRA Model & ComfyUI Workflow | Virtual Try-on | ![Sample Result](./images/ComfyUI_temp_ditfb_00016_.jpeg) |
|
59 |
+
| 3. [Flux In Context - visual identity Lora in Comfy](https://civitai.com/articles/8779) | ComfyUI Workflow | Visual Identity Transfer | ![Sample Result](./images/ComfyUI_00026_.jpeg) |
|
60 |
+
| 4. [Workflows Flux In Context Lora For Product Design](https://civitai.com/models/933018/workflows-flux-in-context-lora-for-product-design) | ComfyUI Workflow | Product Design, Role Play, and More | ![Sample Result](./images/ComfyUI_temp_opjou_00016_.jpeg) |
|
61 |
+
| 5. [Flux Product Design - In Context Lora](https://civitai.com/models/933026/flux-product-design-in-context-lora) | LoRA Model & ComfyUI Workflow | Product Design | ![Sample Result](./images/2024-11-10-002611_0.jpeg) |
|
62 |
+
| 6. [In Context lora + Character story generator + flux+ shichen](https://civitai.com/models/951357/in-context-lora-character-story-generator-flux-shichen) | ComfyUI Workflow | Character Movie Story Generator | ![Sample Result](./images/role2story.jpeg) |
|
63 |
+
| 7. [In- Context-Lora|Cute 4koma 可爱四格漫画](https://civitai.com/models/947702/in-context-loracute-4koma) | LoRA Model & ComfyUI Workflow | Comic Strip Generation | ![Sample Result](./images/ComfyUI_00098_.jpeg) |
|
64 |
+
| 8. [Creative Effects & Design LoRA Pack (In-Context LORA)](https://civitai.com/models/929592/creative-effects-and-design-lora-pack-in-context-lora) | LoRA Model & ComfyUI Workflow | Movie-Shot Generation and More | ![Sample Result](./images/film-storyboard-1.jpeg) |
|
65 |
+
|
66 |
+
We extend our heartfelt thanks to all contributors for their exceptional work in advancing the IC-LoRA ecosystem.
|
67 |
+
|
68 |
## LICENSE
|
69 |
|
70 |
This model hub uses FLUX as the base model. Users must comply with FLUX's license when using this code. Please refer to [FLUX's License](https://github.com/black-forest-labs/flux/tree/main/model_licenses) for more details.
|
71 |
|
72 |
## Citation
|
73 |
|
74 |
+
If you find this work useful in your research, please consider citing:
|
75 |
|
76 |
```bibtex
|
77 |
@article{lhhuang2024iclora,
|
|
|
82 |
}
|
83 |
```
|
84 |
|
85 |
+
```bibtex
|
86 |
+
@article{lhhuang2024iclora,
|
87 |
+
title={Group Diffusion Transformers are Unsupervised Multitask Learners},
|
88 |
+
author={Huang, Lianghua and Wang, Wei and Wu, Zhi-Fan and Dou, Huanzhang and Shi, Yupeng and Feng, Yutong and Liang, Chen and Liu, Yu and Zhou, Jingren},
|
89 |
+
journal={arXiv preprint arxiv:2410.15027},
|
90 |
+
year={2024}
|
91 |
+
}
|
92 |
+
```
|
93 |
+
|
94 |
## Download model
|
95 |
|
96 |
Weights for these models are available in Safetensors format.
|