Update README.md (#14)

- Update README.md (a7dd3f3a8c51c0c43f9645e48754163b5795b5c1)
- Update README.md (f22c97131d1e6698d1d09c8e0f9937424284e73a)
- Update README.md (39999b3ebd9123a62bc541d9d2b5a161d82e3e86)


Co-authored-by: Yoach Lacombe <[email protected]>

README.md

@@ -103,15 +103,48 @@ inference: false
 
 We are open-sourcing our Conformer-based [W2v-BERT 2.0 speech encoder](#w2v-bert-20-speech-encoder) as described in Section 3.2.1 of the [paper](https://arxiv.org/pdf/2312.05187.pdf), which is at the core of our Seamless models.
 
+This model was pre-trained on 4.5M hours of unlabeled audio data covering more than 143 languages. It requires finetuning to be used for downstream tasks such as Automatic Speech Recognition (ASR), or Audio Classification.
+
 | Model Name        | #params | checkpoint                                                                                                                                                                                                                                                                                                                                                                 |
 | ----------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | W2v-BERT 2.0 | 600M    | [checkpoint](https://huggingface.co/reach-vb/conformer-shaw/resolve/main/conformer_shaw.pt)
 
-Scaling data size for self-supervised pre-training has been empirically proven to be a relatively cheap, yet effective way to improve speech representation quality (Zhang et al., 2023a). Following such direction, we continued to add more unlabeled speech data, increasing the amount of our pre-training data from 1M hours (Seamless Communication et al., 2023) to approximately 4.5M hours.
-Besides leveraging more pre-training data, we removed the random-projection quantizer (RPQ) (Chiu et al., 2022) and its associated loss previously incorporated in SeamlessM4T v1 (Seamless Communication et al., 2023).4 Akin to v1, the v2 w2v-BERT 2.0 comprises 24 Conformer layers (Gulati et al., 2020) with approximately 600M parameters and the same pre-training hyperparameters.
+**This model and its training are supported by 🤗 Transformers, more on it in the [docs](https://huggingface.co/docs/transformers/main/en/model_doc/wav2vec2-bert).**
+
+
+# 🤗 Transformers usage
+
+This is a bare checkpoint without any modeling head, and thus requires finetuning to be used for downstream tasks such as ASR. You can however use it to extract audio embeddings from the top layer with this code snippet:
+
+```python
+from transformers import AutoFeatureExtractor, Wav2Vec2BertModel
+import torch
+from datasets import load_dataset
+
+dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
+dataset = dataset.sort("id")
+sampling_rate = dataset.features["audio"].sampling_rate
+
+processor = AutoProcessor.from_pretrained("facebook/w2v-bert-2.0")
+model = Wav2Vec2BertModel.from_pretrained("facebook/w2v-bert-2.0")
+
+# audio file is decoded on the fly
+inputs = processor(dataset[0]["audio"]["array"], sampling_rate=sampling_rate, return_tensors="pt")
+with torch.no_grad():
+    outputs = model(**inputs)
+```
 
+To learn more about the model use, refer to the following resources:
+- [its docs](https://huggingface.co/docs/transformers/main/en/model_doc/wav2vec2-bert)
+- [a blog post showing how to fine-tune it on Mongolian ASR](https://huggingface.co/blog/fine-tune-w2v2-bert)
+- [a training script example](https://github.com/huggingface/transformers/blob/main/examples/pytorch/speech-recognition/run_speech_recognition_ctc.py)
 
-Here's how you should do a forward pass through the speech encoder:
+
+# Seamless Communication usage
+
+This model can be used in [Seamless Communication](https://github.com/facebookresearch/seamless_communication), where it was released.
+ 
+Here's how to make a forward pass through the voice encoder, after having completed the [installation steps](https://github.com/facebookresearch/seamless_communication?tab=readme-ov-file#installation):
 
 ```python
 import torch
@@ -148,4 +181,4 @@ seqs, padding_mask = get_seqs_and_padding_mask(src)
 with torch.inference_mode():
   seqs, padding_mask = model.encoder_frontend(seqs, padding_mask)
   seqs, padding_mask = model.encoder(seqs, padding_mask)
-```
+```