Upload LLaVA-LoRA model

README.md

@@ -0,0 +1,34 @@
+
+# LLaVA-LoRA Adapter
+
+This is a LoRA adapter for the LLaVA model, fine-tuned for spatial description tasks.
+
+## Base Model
+This adapter is trained on top of [llava-hf/llava-1.5-7b-hf](https://huggingface.co/llava-hf/llava-1.5-7b-hf).
+
+## Training
+The model was fine-tuned using LoRA with the following configuration:
+- Rank: 8
+- Alpha: 32
+- Target modules: q_proj, v_proj, k_proj
+- Dataset: PersReFex validation set
+
+## Usage
+
+```python
+from peft import PeftModel
+from transformers import AutoProcessor, LlavaForConditionalGeneration
+
+# Load base model
+base_model = LlavaForConditionalGeneration.from_pretrained(
+    "llava-hf/llava-1.5-7b-hf",
+    torch_dtype=torch.bfloat16
+)
+processor = AutoProcessor.from_pretrained("llava-hf/llava-1.5-7b-hf")
+
+# Load LoRA adapter
+model = PeftModel.from_pretrained(
+    base_model,
+    "ZinengTang/llava-lora-spatial"
+)
+```

adapter_config.json

@@ -0,0 +1,13 @@
+{
+  "base_model_name_or_path": "llava-hf/llava-1.5-7b-hf",
+  "task_type": "CAUSAL_LM",
+  "inference_mode": false,
+  "r": 8,
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "target_modules": [
+    "q_proj",
+    "v_proj",
+    "k_proj"
+  ]
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20c637b9b990a03c8c1b93152fb6bdcd6796cd6bd44628521b0e663708b59ded
+size 29936104