Some weights of Videollama2Qwen2ForCausalLM were not initialized from the model checkpoint at ./VideoLLaMA2.1-7B-AV and are newly initialized:
I have the following question, can you help me with that? Why can't I load the parameters of the audio tower successfully? Which file is the model parameter of Audio Tower and which file should I place it in?
Some weights of Videollama2Qwen2ForCausalLM were not initialized from the model checkpoint at ./VideoLLaMA2.1-7B-AV and are newly initialized: ['model.audio_tower.encoder.pos_conv.0.weight_g', 'model.audio_tower.encoder.pos_conv.0.weight_v']
do you deal with that problem? i meet the same problem
At present, there is none, but it can actually be inferred, and the inference effect is the same as the inference result given by the official demo website, so it should not affect it.
After infering, there no results, only "Some weights of Videollama2Qwen2ForCausalLM were not initialized from the model checkpoint at DAMO-NLP-SG/VideoLLaMA2.1-7B-AV and are newly initialized: ['model.audio_tower.encoder.pos_conv.0.weight_g', 'model.audio_tower.encoder.pos_conv.0.weight_v']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
I've also encountered this problem with you, but I'm trying to figure it out.
”BEATs_iter3_plus_AS2M_finetuned_on_AS2M_cpt2.pt “
See if there is this file
i put the .pt in the same place, but it encounter the same problem.
Ensure consistency in dependencies and requirements.txt files in your environment，It is recommended to create a new environment.
Thanks very much! i tried to rebuild the environment.