Image-Text-to-Text
PEFT
Safetensors
Sogand Salehi commited on
Commit
156100b
·
verified ·
1 Parent(s): 8d1637c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -56
README.md CHANGED
@@ -1,63 +1,45 @@
1
  ---
2
- {}
 
 
 
3
  ---
4
 
5
- # ViPer: Metric
6
- GitHub: https://github.com/sogandstorme/ViPer_Personalization
7
 
8
- ## Example
9
 
10
- ```bash
11
- git clone https://github.com/sogandstorme/ViPer_Personalization.git
12
- cd ViPer_Personalization
13
- ```
 
 
 
 
 
 
 
 
 
14
 
15
  ```python
16
- from metric import (
17
- set_device,
18
- load_context_images,
19
- initialize_processor_and_model,
20
- calculate_score
21
- )
22
-
23
- # Ensure that the number of liked and disliked images are the same
24
-
25
- negative_image_paths = [
26
- "disliked/0.png",
27
- "disliked/1.png",
28
- "disliked/2.png",
29
- "disliked/3.png",
30
- "disliked/4.png",
31
- "disliked/5.png",
32
- "disliked/6.png",
33
- "disliked/7.png",
34
- "disliked/8.png",
35
- ]
36
-
37
- positive_image_paths = [
38
- "liked/0.png",
39
- "liked/1.png",
40
- "liked/2.png",
41
- "liked/3.png",
42
- "liked/4.png",
43
- "liked/5.png",
44
- "liked/6.png",
45
- "liked/7.png",
46
- "liked/8.png",
47
- ]
48
-
49
- # Specify the address of the query image
50
- query_image = "query.png"
51
-
52
- device = set_device("cuda:0")
53
-
54
- # Initialize processor and model
55
- device = set_device("cuda:0")
56
- context_images = load_context_images(negative_image_paths, positive_image_paths)
57
- processor, model = initialize_processor_and_model(device)
58
-
59
- # Calculate and print score
60
- score = calculate_score(processor, model, context_images, query_image)
61
-
62
- print(score)
63
- ```
 
1
  ---
2
+ license: other
3
+ license_name: sample-code-license
4
+ license_link: LICENSE
5
+ library_name: viper-vpe
6
  ---
7
 
8
+ # ViPer: Visual Personalization of Generative Models via Individual Preference Learning
 
9
 
10
+ *Tuning-free framework for personalized image generation*
11
 
12
+ [`Website`](https://viper.epfl.ch) | [`GitHub`](https://github.com/EPFL-VILAB/ViPer) | [`BibTeX`](#citation)
13
+
14
+
15
+ We introduce **ViPer**, a method that personalizes the output of generative models to align with different users’ visual preferences for the same prompt. This is done via a one-time capture of the user’s general preferences and conditioning the generative model on them without the need for engineering detailed prompts.
16
+
17
+
18
+ ## Installation
19
+ For install instructions, please see https://github.com/EPFL-VILAB/ViPer.
20
+
21
+
22
+ ## Usage
23
+
24
+ This model can be loaded from Hugging Face Hub as follows:
25
 
26
  ```python
27
+ from transformers import AutoProcessor, BitsAndBytesConfig, AutoModelForVision2Seq
28
+ from peft import PeftModel
29
+
30
+ model = AutoModelForVision2Seq.from_pretrained("HuggingFaceM4/idefics2-8b").to(device)
31
+ model = PeftModel.from_pretrained(model, "EPFL-VILAB/Metric-ViPer").to(device)
32
+ ```
33
+
34
+ Please see https://github.com/EPFL-VILAB/ViPer for more detailed instructions.
35
+
36
+ ## Citation
37
+
38
+ If you find this repository helpful, please consider citing our work:
39
+ ```
40
+ TODO
41
+ ```
42
+
43
+ ## License
44
+
45
+ Licensed under the Apache License, Version 2.0. See [LICENSE](https://github.com/sogandstorme/ViPer_Personalization/blob/main/LICENSE) for details.