metadata

license: mit
language:
  - en
pipeline_tag: audio-classification
tags:
  - wavlm
  - msp-podcast
  - emotion-recognition
  - audio
  - speech
  - valence
  - arousal
  - dominance
  - lucas

The model was trained on MSP-Podcast for the Odyssey 2024 Emotion Recognition competition baseline
This particular model is the multi-attributed based model which predict arousal, dominance and valence in a range of approximately 0...1.

For more details: demo, paper/soon and GitHub.

@misc{SER_Challenge_2024,
  author = {Goncalves, Lucas and Salman, Ali and Reddy, Abinay and Velazquez, Laureano Moro and Thebaud, Thomas and Garcia, Leibny Paola and Dehak, Najim and Sisman, Berrak and Busso, Carlos},
  title = {Odyssey 2024 - Emotion Recognition Challenge},
  year = {2024},
  publisher = {GitHub},
  journal = {MSP-Podcast Challenge},
  howpublished = {\url{https://github.com/MSP-UTD/MSP-Podcast_Challenge}},
}

Usage

from transformers import AutoModelForAudioClassification
import librosa, torch

#load model
model = AutoModelForAudioClassification.from_pretrained("3loi/SER-Odyssey-Baseline-WavLM-Multi-Attributes", trust_remote_code=True)

#get mean/std
mean = model.config.mean
std = model.config.std


#load an audio file
audio_path = "/path/to/audio.wav"
raw_wav, _ = librosa.load(audio_path, sr=model.config.sampling_rate)

#normalize the audio by mean/std
norm_wav = (raw_wav - mean) / (std+0.000001)

#generate the mask
mask = torch.ones(1, len(norm_wav))
wavs = torch.tensor(norm_wav).unsqueeze(0)


#predict
with torch.no_grad():
    pred = model(wavs, mask)

print(model.config.id2label) 
print(pred)
#{0: 'arousal', 1: 'dominance', 2: 'valence'}
#tensor([[0.3670, 0.4553, 0.4240]])