Feature Extraction
music
sander-wood commited on
Commit
7ae264c
·
verified ·
1 Parent(s): 22ed868

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -11
README.md CHANGED
@@ -103,13 +103,19 @@ tags:
103
  - music
104
  ---
105
  # **CLaMP 3: Universal Music Information Retrieval Across Unaligned Modalities and Unseen Languages**
 
 
 
 
 
 
106
 
107
  <p align="center">
108
  <img src="overview.png" alt="CLaMP 3 Overview" width="50%">
109
  </p>
110
 
111
  ## **Overview**
112
- CLaMP 3 is a **multimodal and multilingual** framework for **music information retrieval (MIR)** that fully supports all major music modalities—**sheet music, audio, and performance signals—along with multilingual text**. Using **contrastive learning**, it aligns these modalities into a **shared representation space**, enabling seamless cross-modal retrieval. Experiments demonstrate that CLaMP 3 **significantly outperforms previous strong baselines**, establishing a new state-of-the-art in multimodal and multilingual MIR.
113
 
114
  ### **Key Features**
115
  - **Multimodal Support:**
@@ -134,22 +140,14 @@ CLaMP 3 supports a **wide range of music research tasks**, including but not lim
134
 
135
  Importantly, these applications are **not restricted to any specific music modality or language**, making CLaMP 3 a powerful tool for **diverse music AI research**.
136
 
137
- ## **Links**
138
- - **CLaMP 3 Demo Page** *(Coming Soon...)*
139
- - **CLaMP 3 Paper** *(Coming Soon...)*
140
- - **[CLaMP 3 Code](https://github.com/sanderwood/clamp3)**
141
- - **[CLaMP 3 Model Weights](https://huggingface.co/sander-wood/clamp3/tree/main)**
142
- - **[M4-RAG Pre-training Dataset](https://huggingface.co/datasets/sander-wood/m4-rag)**
143
- - **[WikiMT-X Evaluation Benchmark](https://huggingface.co/datasets/sander-wood/wikimt-x)**
144
-
145
- > **Note:** Ensure the model weights are placed in the `code/` folder, and verify the **configuration hyperparameters** before use.
146
-
147
  ## **Repository Structure**
148
  - **[code/](https://github.com/sanderwood/clamp3/tree/main/code)** → Training & feature extraction scripts.
149
  - **[classification/](https://github.com/sanderwood/clamp3/tree/main/classification)** → Linear classification training and prediction.
150
  - **[preprocessing/](https://github.com/sanderwood/clamp3/tree/main/preprocessing)** → Convert data into **Interleaved ABC, MTF, or MERT-extracted features**.
151
  - **[retrieval/](https://github.com/sanderwood/clamp3/tree/main/retrieval)** → Semantic search, retrieval evaluation, and similarity calculations.
152
 
 
 
153
  ## **Getting Started**
154
  ### **Environment Setup**
155
  To set up the environment for CLaMP 3, run:
 
103
  - music
104
  ---
105
  # **CLaMP 3: Universal Music Information Retrieval Across Unaligned Modalities and Unseen Languages**
106
+ [![Demo](https://img.shields.io/badge/CLaMP%203%20Demo-Coming%20Soon-lightgrey?style=for-the-badge&logo=gradio)](#)
107
+ [![Paper](https://img.shields.io/badge/CLaMP%203%20Paper-Coming%20Soon-lightgrey?style=for-the-badge&logo=arxiv)](#)
108
+ [![GitHub](https://img.shields.io/badge/Code-GitHub-181717?style=for-the-badge&logo=github)](https://github.com/sanderwood/clamp3)
109
+ [![Hugging Face](https://img.shields.io/badge/Model%20Weights-Hugging%20Face-ffcc00?style=for-the-badge&logo=huggingface)](https://huggingface.co/sander-wood/clamp3/tree/main)
110
+ [![Dataset](https://img.shields.io/badge/M4--RAG%20Pretraining%20Dataset-Hugging%20Face-ffcc00?style=for-the-badge&logo=huggingface)](https://huggingface.co/datasets/sander-wood/m4-rag)
111
+ [![Benchmark](https://img.shields.io/badge/WikiMT--X%20Evaluation%20Benchmark-Hugging%20Face-ffcc00?style=for-the-badge&logo=huggingface)](https://huggingface.co/datasets/sander-wood/wikimt-x)
112
 
113
  <p align="center">
114
  <img src="overview.png" alt="CLaMP 3 Overview" width="50%">
115
  </p>
116
 
117
  ## **Overview**
118
+ CLaMP 3 is a multimodal and multilingual framework for music information retrieval (MIR) that supports all major music formats—sheet music, audio, and performance signals—along with multilingual text. It is trained on 27 languages and can generalize to support 100 languages. Using contrastive learning, CLaMP 3 aligns these different formats into a shared representation space, making cross-modal retrieval seamless. Experiments show that it significantly outperforms previous strong baselines, setting a new state-of-the-art in multimodal and multilingual MIR.
119
 
120
  ### **Key Features**
121
  - **Multimodal Support:**
 
140
 
141
  Importantly, these applications are **not restricted to any specific music modality or language**, making CLaMP 3 a powerful tool for **diverse music AI research**.
142
 
 
 
 
 
 
 
 
 
 
 
143
  ## **Repository Structure**
144
  - **[code/](https://github.com/sanderwood/clamp3/tree/main/code)** → Training & feature extraction scripts.
145
  - **[classification/](https://github.com/sanderwood/clamp3/tree/main/classification)** → Linear classification training and prediction.
146
  - **[preprocessing/](https://github.com/sanderwood/clamp3/tree/main/preprocessing)** → Convert data into **Interleaved ABC, MTF, or MERT-extracted features**.
147
  - **[retrieval/](https://github.com/sanderwood/clamp3/tree/main/retrieval)** → Semantic search, retrieval evaluation, and similarity calculations.
148
 
149
+ > **Note:** Ensure the model weights are placed in the `code/` folder, and verify the **configuration hyperparameters** before use.
150
+
151
  ## **Getting Started**
152
  ### **Environment Setup**
153
  To set up the environment for CLaMP 3, run: