Text-to-Image
Diffusers
lora
template:diffusion-lora
lhhuang commited on
Commit
3b4d5f3
1 Parent(s): b74ddad

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +43 -1
README.md CHANGED
@@ -9,12 +9,28 @@ instance_prompt: null
9
  license: mit
10
  ---
11
  📢 [[Project Page](https://ali-vilab.github.io/In-Context-LoRA-Page/)] [[Github Repo](https://github.com/ali-vilab/In-Context-LoRA)] [[Paper](https://arxiv.org/abs/2410.23775)]
 
 
 
 
12
  # Model Summary
13
 
14
  In-Context LoRA fine-tunes text-to-image models (*e.g.,* [FLUX](https://huggingface.co/black-forest-labs/FLUX.1-dev)) to generate image sets with customizable intrinsic relationships, optionally conditioned on another set using SDEdit. It can be adapted to a wide range of tasks
15
 
16
  This model hub includes In-Context LoRA models across 10 tasks. [MODEL ZOO](#model-zoo) details these models and their recommend settings. For more details on how these models are trained, please refer to our [paper](https://arxiv.org/abs/2410.23775).
17
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  ## MODEL ZOO
19
 
20
  Below lists 10 In-Context LoRA models and their recommend settings.
@@ -32,13 +48,30 @@ Below lists 10 In-Context LoRA models and their recommend settings.
32
  | **9. Sparklers Visual Effect** | [`sparklers-visual-effect.safetensors`](https://huggingface.co/ali-vilab/In-Context-LoRA/blob/main/sparklers-visual-effect.safetensors) | `width: 960, height: 1088` | `[REAL-SPARKLERS-OVERLAYS] The two-part image vividly illustrates a woodland proposal transformed by sparkler overlays; [TOP] the first panel depicts a man kneeling on one knee with an engagement ring before his partner in a forest clearing at dusk, with warm, natural lighting, [BOTTOM] while the second panel introduces glowing sparklers that form a heart shape around the couple, amplifying the romance and joy of the moment.` |
33
  | **10. Visual Identity Design** | [`visual-identity-design.safetensors`](https://huggingface.co/ali-vilab/In-Context-LoRA/blob/main/visual-identity-design.safetensors) | `width: 1472, height: 1024` | `The two-panel image showcases the joyful identity of a produce brand, with the left panel showing a smiling pineapple graphic and the brand name “Fresh Tropic” in a fun, casual font on a light aqua background; [LEFT] while the right panel translates the design onto a reusable shopping tote with the pineapple logo in black, held by a person in a market setting, emphasizing the brand’s approachable and eco-friendly vibe.` |
34
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
35
  ## LICENSE
36
 
37
  This model hub uses FLUX as the base model. Users must comply with FLUX's license when using this code. Please refer to [FLUX's License](https://github.com/black-forest-labs/flux/tree/main/model_licenses) for more details.
38
 
39
  ## Citation
40
 
41
- If you find this model hub useful in your research, please consider citing:
42
 
43
  ```bibtex
44
  @article{lhhuang2024iclora,
@@ -49,6 +82,15 @@ If you find this model hub useful in your research, please consider citing:
49
  }
50
  ```
51
 
 
 
 
 
 
 
 
 
 
52
  ## Download model
53
 
54
  Weights for these models are available in Safetensors format.
 
9
  license: mit
10
  ---
11
  📢 [[Project Page](https://ali-vilab.github.io/In-Context-LoRA-Page/)] [[Github Repo](https://github.com/ali-vilab/In-Context-LoRA)] [[Paper](https://arxiv.org/abs/2410.23775)]
12
+ # 🔥 Latest News
13
+
14
+ - **[2024-11-16]** 🌟 The community continues to innovate with IC-LoRA! Exciting projects include models, ComfyUI nodes and workflows for **Virtual Try-on, Product Design, Object Mitigation, Role Play**, and more. Explore their creations in **[Community Creations Using IC-LoRA](#community-creations-using-ic-lora)**. Huge thanks to all contributors for their incredible efforts!
15
+
16
  # Model Summary
17
 
18
  In-Context LoRA fine-tunes text-to-image models (*e.g.,* [FLUX](https://huggingface.co/black-forest-labs/FLUX.1-dev)) to generate image sets with customizable intrinsic relationships, optionally conditioned on another set using SDEdit. It can be adapted to a wide range of tasks
19
 
20
  This model hub includes In-Context LoRA models across 10 tasks. [MODEL ZOO](#model-zoo) details these models and their recommend settings. For more details on how these models are trained, please refer to our [paper](https://arxiv.org/abs/2410.23775).
21
 
22
+ ## Key Idea
23
+
24
+ The core concept of IC-LoRA is to **concatenate** both condition and target images into a single composite image while using **Natural Language** to define the task. This approach enables seamless adaptation to a wide range of applications.
25
+
26
+ ## Features
27
+
28
+ - **Task-Agnostic Framework**: IC-LoRA serves as a general framework, but it requires task-specific fine-tuning for diverse applications.
29
+ - **Customizable Image-Set Generation**: You can fine-tune text-to-image models to **generate image sets** with customizable intrinsic relationships.
30
+ - **Condition on Image-Set**: You can also **condition the generation of a set of images on another set of images**, enabling a wide range of controllable generation applications.
31
+
32
+ For more detailed information and examples, please read our [Paper](https://arxiv.org/abs/2410.23775) or visit our [Project Page](https://ali-vilab.github.io/In-Context-LoRA-Page/).
33
+
34
  ## MODEL ZOO
35
 
36
  Below lists 10 In-Context LoRA models and their recommend settings.
 
48
  | **9. Sparklers Visual Effect** | [`sparklers-visual-effect.safetensors`](https://huggingface.co/ali-vilab/In-Context-LoRA/blob/main/sparklers-visual-effect.safetensors) | `width: 960, height: 1088` | `[REAL-SPARKLERS-OVERLAYS] The two-part image vividly illustrates a woodland proposal transformed by sparkler overlays; [TOP] the first panel depicts a man kneeling on one knee with an engagement ring before his partner in a forest clearing at dusk, with warm, natural lighting, [BOTTOM] while the second panel introduces glowing sparklers that form a heart shape around the couple, amplifying the romance and joy of the moment.` |
49
  | **10. Visual Identity Design** | [`visual-identity-design.safetensors`](https://huggingface.co/ali-vilab/In-Context-LoRA/blob/main/visual-identity-design.safetensors) | `width: 1472, height: 1024` | `The two-panel image showcases the joyful identity of a produce brand, with the left panel showing a smiling pineapple graphic and the brand name “Fresh Tropic” in a fun, casual font on a light aqua background; [LEFT] while the right panel translates the design onto a reusable shopping tote with the pineapple logo in black, held by a person in a market setting, emphasizing the brand’s approachable and eco-friendly vibe.` |
50
 
51
+ ## Community Creations Using IC-LoRA
52
+
53
+ We are thrilled to showcase the community's innovative projects leveraging In-Context LoRA (IC-LoRA). If you have additional recommendations or projects to share, **please don't hesitate to send a [Pull Request](https://github.com/ali-vilab/In-Context-LoRA/pulls)!**
54
+
55
+ | Project Name | Type | Supported Tasks | Sample Results |
56
+ |--------------|----------------------|---------------------------------------------------------------------------------|----------------|
57
+ | 1. [Comfyui_Object_Migration](https://github.com/TTPlanetPig/Comfyui_Object_Migration) | ComfyUI Node & Workflow & LoRA Model | Clothing Migration, Cartoon Clothing to Realism, and More | ![Sample Result](./images/386534865-9612cf8a-858d-4684-819e-7b97981d993c.png) |
58
+ | 2. [Flux Simple Try On - In Context Lora](https://civitai.com/models/950111/flux-simple-try-on-in-context-lora) | LoRA Model & ComfyUI Workflow | Virtual Try-on | ![Sample Result](./images/ComfyUI_temp_ditfb_00016_.jpeg) |
59
+ | 3. [Flux In Context - visual identity Lora in Comfy](https://civitai.com/articles/8779) | ComfyUI Workflow | Visual Identity Transfer | ![Sample Result](./images/ComfyUI_00026_.jpeg) |
60
+ | 4. [Workflows Flux In Context Lora For Product Design](https://civitai.com/models/933018/workflows-flux-in-context-lora-for-product-design) | ComfyUI Workflow | Product Design, Role Play, and More | ![Sample Result](./images/ComfyUI_temp_opjou_00016_.jpeg) |
61
+ | 5. [Flux Product Design - In Context Lora](https://civitai.com/models/933026/flux-product-design-in-context-lora) | LoRA Model & ComfyUI Workflow | Product Design | ![Sample Result](./images/2024-11-10-002611_0.jpeg) |
62
+ | 6. [In Context lora + Character story generator + flux+ shichen](https://civitai.com/models/951357/in-context-lora-character-story-generator-flux-shichen) | ComfyUI Workflow | Character Movie Story Generator | ![Sample Result](./images/role2story.jpeg) |
63
+ | 7. [In- Context-Lora|Cute 4koma 可爱四格漫画](https://civitai.com/models/947702/in-context-loracute-4koma) | LoRA Model & ComfyUI Workflow | Comic Strip Generation | ![Sample Result](./images/ComfyUI_00098_.jpeg) |
64
+ | 8. [Creative Effects & Design LoRA Pack (In-Context LORA)](https://civitai.com/models/929592/creative-effects-and-design-lora-pack-in-context-lora) | LoRA Model & ComfyUI Workflow | Movie-Shot Generation and More | ![Sample Result](./images/film-storyboard-1.jpeg) |
65
+
66
+ We extend our heartfelt thanks to all contributors for their exceptional work in advancing the IC-LoRA ecosystem.
67
+
68
  ## LICENSE
69
 
70
  This model hub uses FLUX as the base model. Users must comply with FLUX's license when using this code. Please refer to [FLUX's License](https://github.com/black-forest-labs/flux/tree/main/model_licenses) for more details.
71
 
72
  ## Citation
73
 
74
+ If you find this work useful in your research, please consider citing:
75
 
76
  ```bibtex
77
  @article{lhhuang2024iclora,
 
82
  }
83
  ```
84
 
85
+ ```bibtex
86
+ @article{lhhuang2024iclora,
87
+ title={Group Diffusion Transformers are Unsupervised Multitask Learners},
88
+ author={Huang, Lianghua and Wang, Wei and Wu, Zhi-Fan and Dou, Huanzhang and Shi, Yupeng and Feng, Yutong and Liang, Chen and Liu, Yu and Zhou, Jingren},
89
+ journal={arXiv preprint arxiv:2410.15027},
90
+ year={2024}
91
+ }
92
+ ```
93
+
94
  ## Download model
95
 
96
  Weights for these models are available in Safetensors format.