|
--- |
|
tags: |
|
- opt_metasq |
|
--- |
|
|
|
# This repo let's you run the following checkpoint using facebookresearch/metaseq. |
|
|
|
Do the following: |
|
|
|
## 1. Install PyTorch |
|
``` |
|
pip3 install torch==1.10.1+cu113 torchvision==0.11.2+cu113 torchaudio==0.10.1+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html |
|
``` |
|
|
|
## 2. Install Megatron |
|
``` |
|
git clone https://github.com/patrickvonplaten/Megatron-LM.git |
|
cd Megatron-LM |
|
pip3 install six regex |
|
pip3 install -e . |
|
``` |
|
## 3. Install fairscale |
|
``` |
|
git clone https://github.com/facebookresearch/fairscale.git |
|
cd fairscale |
|
git checkout prefetch_fsdp_params_simple |
|
pip3 install -e . |
|
``` |
|
## 4. Install metaseq |
|
``` |
|
git clone https://github.com/patrickvonplaten/metaseq.git |
|
cd metaseq |
|
pip3 install -e . |
|
``` |
|
|
|
## 5. Clone this repo (click top right on "How to clone") |
|
|
|
## 6. Run the following: |
|
|
|
```bash |
|
cd <path/to/cloned/repo> |
|
bash run.sh |
|
``` |
