Suggestion to add Megatron to PYTHONPATH
Browse filesExplains how to add fix a ModuleNotFoundError by adding Megatron to the PYTHONPATH
README.md
CHANGED
@@ -118,6 +118,22 @@ wget --content-disposition https://api.ngc.nvidia.com/v2/models/nvidia/megatron_
|
|
118 |
python src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py megatron_lm_345m_v0.0.zip
|
119 |
```
|
120 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
121 |
3. Fetch missing files
|
122 |
```
|
123 |
git clone https://huggingface.co/nvidia/megatron-gpt2-345m/
|
|
|
118 |
python src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py megatron_lm_345m_v0.0.zip
|
119 |
```
|
120 |
|
121 |
+
As explained by Stas Bekman in [PR #14956](https://github.com/huggingface/transformers/pull/14956), if when running this conversion
|
122 |
+
script and you're getting an exception:
|
123 |
+
```
|
124 |
+
ModuleNotFoundError: No module named 'megatron.model.enums'
|
125 |
+
```
|
126 |
+
you need to tell python where to find the clone of Megatron-LM, e.g.:
|
127 |
+
```
|
128 |
+
cd /tmp
|
129 |
+
git clone https://github.com/NVIDIA/Megatron-LM
|
130 |
+
PYTHONPATH=/tmp/Megatron-LM python src/transformers/models/megatron_bert/convert_megatron_bert_checkpoint.py ...
|
131 |
+
```
|
132 |
+
Or, if you already have it cloned elsewhere, simply adjust the path to the existing path.
|
133 |
+
|
134 |
+
If the training was done using a Megatron-LM fork, e.g. [Megatron-DeepSpeed](https://github.com/microsoft/Megatron-DeepSpeed/) then
|
135 |
+
you may need to have that one in your path, i.e., /path/to/Megatron-DeepSpeed.
|
136 |
+
|
137 |
3. Fetch missing files
|
138 |
```
|
139 |
git clone https://huggingface.co/nvidia/megatron-gpt2-345m/
|