Create README.md

README.md

@@ -0,0 +1,61 @@
+# Don’t Judge Before You CLIP: Memorability Prediction Model
+
+This model is part of our paper:  
+*"Don’t Judge Before You CLIP: A Unified Approach for Perceptual Tasks"*  
+It was trained on the *LaMem dataset* to predict image memorability scores.
+
+## Model Overview
+
+Visual perceptual tasks, such as image memorability prediction, aim to estimate how humans perceive and interpret images. Unlike objective tasks (e.g., object recognition), these tasks rely on subjective human judgment, making labeled data scarce.
+
+Our approach leverages *CLIP* as a prior for perceptual tasks, inspired by cognitive research suggesting that CLIP implicitly captures human biases, emotions, and preferences. We fine-tune CLIP minimally using *LoRA* to adapt it to each specific task.
+
+## Training Details
+
+- *Dataset*: [LaMem](http://memorability.csail.mit.edu/download.html) (Large-Scale Image Memorability)
+- *Architecture*: CLIP Vision Encoder (ViT-L/14) with *LoRA adaptation*
+- *Loss Function*: Mean Squared Error (MSE) Loss for memorability prediction
+- *Optimizer*: AdamW
+- *Learning Rate*: 0.00005
+- *Batch Size*: 32
+
+## Performance
+
+The model was trained on the *LaMem dataset* and exhibits *state-of-the-art generalization* to the *THINGS memorability dataset*.  
+For more models and results on the *five common splits* of LaMem, please refer to our paper.
+
+## Usage
+
+To use the model for inference:
+
+```python
+from torchvision import transforms
+import torch
+from PIL import Image
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+# Load model
+model = torch.load("lamem_all_clip_Lora_16.0R_8.0alphaLora_32_batch_0.00005_lossmse_headmlp.pth").to(device).eval()
+
+# Load an image
+image = Image.open("image_path.jpg").convert("RGB")
+
+# Preprocess and predict
+def Mem_augmentations():
+    transform = transforms.Compose([
+    transforms.Resize(224),
+    transforms.CenterCrop(size=(224,224)),  
+    transforms.ToTensor(),
+    # Note: The model normalizes the image inside the forward pass
+    # using mean = (0.48145466, 0.4578275, 0.40821073) and 
+    # std = (0.26862954, 0.26130258, 0.27577711)
+])
+    return transform
+
+image = Mem_augmentations()(image).unsqueeze(0).to(device)
+
+with torch.no_grad():
+    mem_score = model(image).item()
+
+print(f"Predicted Memorability Score: {mem_score:.4f}")