Image-Text-to-Text
PEFT
Safetensors
miaw1419 commited on
Commit
34a44be
·
verified ·
1 Parent(s): 5ac4f8d

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +62 -0
README.md ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ {}
3
+ ---
4
+
5
+ # Metric
6
+ GitHub: https://github.com/sogandstorme/ViPer_Personalization
7
+
8
+ ## Example
9
+
10
+ To use ViPer, start by cloning it from our GitHub page. For effective personalization, we recommend commenting on at least eight images.
11
+
12
+ ```bash
13
+ git clone https://github.com/sogandstorme/ViPer_Personalization.git
14
+ cd ViPer_Personalization
15
+ ```
16
+
17
+ ```python
18
+ from ViPer import (
19
+ set_device,
20
+ load_images,
21
+ initialize_processor_and_model,
22
+ prepare_prompt_and_inputs,
23
+ generate_texts,
24
+ extract_features,
25
+ initialize_pipelines,
26
+ generate_images
27
+ )
28
+
29
+ # Ensure that the order of the comments matches the path of the images they refer to.
30
+
31
+ comments = [
32
+ "These are beautiful, intricate patterns. Very elegant, and the teal blue colors are lovely. I love the flowing lines.",
33
+ "The colors here don't quite work for me. They feel a bit unmatched and artificial. The concept also seems a bit boring and artificial to me.",
34
+ ]
35
+
36
+ image_paths = [
37
+ "/images/6.png",
38
+ "/images/9.png"
39
+ ]
40
+
41
+ prompts = [
42
+ "Whimsical tea party in a bioluminescent forest",
43
+ "Tiny houses on top of each other above clouds"
44
+ ]
45
+
46
+ output_dir = "results/"
47
+
48
+ device = set_device("cuda:0")
49
+
50
+ # Initialize processor, model and inputs
51
+ images = load_images(image_paths)
52
+ processor, model = initialize_processor_and_model(device)
53
+ inputs = prepare_prompt_and_inputs(processor, images, comments)
54
+
55
+ # Generate and extract vp
56
+ generated_texts = generate_texts(processor, model, inputs)
57
+ vp_pos, vp_neg = extract_features(generated_texts)
58
+
59
+ # Initialize pipelines and generate images
60
+ pipe, refiner = initialize_pipelines(device)
61
+ generate_images(pipe, refiner, prompts, vp_pos, vp_neg, output_dir)
62
+ ```