Update README.md

README.md

@@ -15,9 +15,31 @@ should probably proofread and complete it, then remove this comment. -->
 
 # paligemma_vqav2
 
-This model is a fine-tuned version of [google/paligemma-3b-pt-224](https://huggingface.co/google/paligemma-3b-pt-224) on the vq_av2 dataset.
+This model is a fine-tuned version of [google/paligemma-3b-pt-224](https://huggingface.co/google/paligemma-3b-pt-224) on a small chunk of vq_av2 dataset.
 
-## Training procedure
+## How to Use
+
+Below is the code to use this model. Also see [inference notebook](https://colab.research.google.com/drive/100IQcvMvGm9y--oelbLfI__eHCoz5Ser?usp=sharing).
+
+```
+from transformers import AutoProcessor, PaliGemmaForConditionalGeneration
+from PIL import Image
+import requests
+
+model_id = "merve/paligemma_vqav2"
+model = PaliGemmaForConditionalGeneration.from_pretrained(model_id)
+processor = AutoProcessor.from_pretrained("google/paligemma-3b-pt-224")
+
+prompt = "What is behind the cat?"
+image_file = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/cat.png?download=true"
+raw_image = Image.open(requests.get(image_file, stream=True).raw)
+
+inputs = processor(prompt, raw_image.convert("RGB"), return_tensors="pt")
+output = model.generate(**inputs, max_new_tokens=20)
+
+print(processor.decode(output[0], skip_special_tokens=True)[len(prompt):])
+# gramophone
+```
 
 ### Training hyperparameters