Update README.md
Browse files
README.md
CHANGED
@@ -103,13 +103,19 @@ tags:
|
|
103 |
- music
|
104 |
---
|
105 |
# **CLaMP 3: Universal Music Information Retrieval Across Unaligned Modalities and Unseen Languages**
|
|
|
|
|
|
|
|
|
|
|
|
|
106 |
|
107 |
<p align="center">
|
108 |
<img src="overview.png" alt="CLaMP 3 Overview" width="50%">
|
109 |
</p>
|
110 |
|
111 |
## **Overview**
|
112 |
-
CLaMP 3 is a
|
113 |
|
114 |
### **Key Features**
|
115 |
- **Multimodal Support:**
|
@@ -134,22 +140,14 @@ CLaMP 3 supports a **wide range of music research tasks**, including but not lim
|
|
134 |
|
135 |
Importantly, these applications are **not restricted to any specific music modality or language**, making CLaMP 3 a powerful tool for **diverse music AI research**.
|
136 |
|
137 |
-
## **Links**
|
138 |
-
- **CLaMP 3 Demo Page** *(Coming Soon...)*
|
139 |
-
- **CLaMP 3 Paper** *(Coming Soon...)*
|
140 |
-
- **[CLaMP 3 Code](https://github.com/sanderwood/clamp3)**
|
141 |
-
- **[CLaMP 3 Model Weights](https://huggingface.co/sander-wood/clamp3/tree/main)**
|
142 |
-
- **[M4-RAG Pre-training Dataset](https://huggingface.co/datasets/sander-wood/m4-rag)**
|
143 |
-
- **[WikiMT-X Evaluation Benchmark](https://huggingface.co/datasets/sander-wood/wikimt-x)**
|
144 |
-
|
145 |
-
> **Note:** Ensure the model weights are placed in the `code/` folder, and verify the **configuration hyperparameters** before use.
|
146 |
-
|
147 |
## **Repository Structure**
|
148 |
- **[code/](https://github.com/sanderwood/clamp3/tree/main/code)** → Training & feature extraction scripts.
|
149 |
- **[classification/](https://github.com/sanderwood/clamp3/tree/main/classification)** → Linear classification training and prediction.
|
150 |
- **[preprocessing/](https://github.com/sanderwood/clamp3/tree/main/preprocessing)** → Convert data into **Interleaved ABC, MTF, or MERT-extracted features**.
|
151 |
- **[retrieval/](https://github.com/sanderwood/clamp3/tree/main/retrieval)** → Semantic search, retrieval evaluation, and similarity calculations.
|
152 |
|
|
|
|
|
153 |
## **Getting Started**
|
154 |
### **Environment Setup**
|
155 |
To set up the environment for CLaMP 3, run:
|
|
|
103 |
- music
|
104 |
---
|
105 |
# **CLaMP 3: Universal Music Information Retrieval Across Unaligned Modalities and Unseen Languages**
|
106 |
+
[](#)
|
107 |
+
[](#)
|
108 |
+
[](https://github.com/sanderwood/clamp3)
|
109 |
+
[](https://huggingface.co/sander-wood/clamp3/tree/main)
|
110 |
+
[](https://huggingface.co/datasets/sander-wood/m4-rag)
|
111 |
+
[](https://huggingface.co/datasets/sander-wood/wikimt-x)
|
112 |
|
113 |
<p align="center">
|
114 |
<img src="overview.png" alt="CLaMP 3 Overview" width="50%">
|
115 |
</p>
|
116 |
|
117 |
## **Overview**
|
118 |
+
CLaMP 3 is a multimodal and multilingual framework for music information retrieval (MIR) that supports all major music formats—sheet music, audio, and performance signals—along with multilingual text. It is trained on 27 languages and can generalize to support 100 languages. Using contrastive learning, CLaMP 3 aligns these different formats into a shared representation space, making cross-modal retrieval seamless. Experiments show that it significantly outperforms previous strong baselines, setting a new state-of-the-art in multimodal and multilingual MIR.
|
119 |
|
120 |
### **Key Features**
|
121 |
- **Multimodal Support:**
|
|
|
140 |
|
141 |
Importantly, these applications are **not restricted to any specific music modality or language**, making CLaMP 3 a powerful tool for **diverse music AI research**.
|
142 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
143 |
## **Repository Structure**
|
144 |
- **[code/](https://github.com/sanderwood/clamp3/tree/main/code)** → Training & feature extraction scripts.
|
145 |
- **[classification/](https://github.com/sanderwood/clamp3/tree/main/classification)** → Linear classification training and prediction.
|
146 |
- **[preprocessing/](https://github.com/sanderwood/clamp3/tree/main/preprocessing)** → Convert data into **Interleaved ABC, MTF, or MERT-extracted features**.
|
147 |
- **[retrieval/](https://github.com/sanderwood/clamp3/tree/main/retrieval)** → Semantic search, retrieval evaluation, and similarity calculations.
|
148 |
|
149 |
+
> **Note:** Ensure the model weights are placed in the `code/` folder, and verify the **configuration hyperparameters** before use.
|
150 |
+
|
151 |
## **Getting Started**
|
152 |
### **Environment Setup**
|
153 |
To set up the environment for CLaMP 3, run:
|