mmaaz60 commited on
Commit
ef1a902
1 Parent(s): 965f32a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -0
README.md CHANGED
@@ -11,6 +11,12 @@ license: mit
11
  This repository features LLaVA v1.5 trained with the Phi-3-mini-3.8B LLM. This integration aims to leverage the strengths of both models to offer advanced vision-language understanding.
12
 
13
 
 
 
 
 
 
 
14
  ## Key Components
15
 
16
  - **Base Large Language Model (LLM):** [Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)
 
11
  This repository features LLaVA v1.5 trained with the Phi-3-mini-3.8B LLM. This integration aims to leverage the strengths of both models to offer advanced vision-language understanding.
12
 
13
 
14
+ ## Training Strategy
15
+ - **Pretraining:** Only Vision-to-Language projector is trained. The rest of the model is frozen.
16
+ - **Fine-tuning:** LLM is LoRA fine-tuned. Only the vision-backbone (CLIP) is kept frozen.
17
+ - **Note:** The repository contains merged weights.
18
+
19
+
20
  ## Key Components
21
 
22
  - **Base Large Language Model (LLM):** [Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)