howto/download_weights.md

Downloading pretrained weights

Except for when you are training from scratch, you will need the pretrained weights from Meta.

Original Meta weights

Download the model weights following the instructions on the official LLaMA repository.

Once downloaded, you should have a folder like this:

checkpoints/llama
├── 7B
│   ├── ...
│   └── consolidated.00.pth
├── 13B
│   ...
└── tokenizer.model

Convert the weights to the Lit-LLaMA format:

python scripts/convert_checkpoint.py --model_size 7B

Note All scripts support argument customization

OpenLLaMA

OpenLM Research has released Apache 2.0 licensed weights obtained by training LLaMA on the 1.2 trillion token open-source RedPajama dataset.

Weights were released in preview on intermediate number of tokens (200B, 300B at the time of writing). In order to get them do:

# Make sure you have git-lfs installed (https://git-lfs.com): git lfs install
git clone https://huggingface.co/openlm-research/open_llama_7b_preview_300bt checkpoints/open-llama/7B

Or if you don't have git-lfs installed:

python scripts/download.py --repo_id openlm-research/open_llama_7b_preview_300bt --local_dir checkpoints/open-llama/7B

Once downloaded, you should have a folder like this:

checkpoints/open-llama/
└── 7B
    └── open_llama_7b_preview_300bt_transformers_weights
        ├── ...
        ├── pytorch_model-00001-of-00002.bin
        ├── pytorch_model-00002-of-00002.bin
        ├── pytorch_model.bin.index.json
        └── tokenizer.model

Convert the weights to the Lit-LLaMA format:

python scripts/convert_hf_checkpoint.py --checkpoint_dir checkpoints/open-llama/7B/open_llama_7b_preview_300bt_transformers_weights --model_size 7B

Note All scripts support argument customization

Once converted, you should have a folder like this:

checkpoints/lit-llama/
├── 7B
│   └── lit-llama.pth
└── tokenizer.model

You are all set. Now you can continue with inference or finetuning.

Try running generate.py to test the imported weights.

Alternative sources

You might find LLaMA weights hosted online in the HuggingFace hub. Beware that this infringes the original weight's license. You could try downloading them by running the following command with a specific repo id:

# Make sure you have git-lfs installed (https://git-lfs.com): git lfs install
git clone REPO_ID checkpoints/hf-llama/7B

Or if you don't have git-lfs installed:

python scripts/download.py --repo_id REPO_ID --local_dir checkpoints/hf-llama/7B

Once downloaded, you should have a folder like this:

checkpoints/hf-llama/
└── 7B
    ├── ...
    ├── pytorch_model-00001-of-00002.bin
    ├── pytorch_model-00002-of-00002.bin
    ├── pytorch_model.bin.index.json
    └── tokenizer.model

Convert the weights to the Lit-LLaMA format:

python scripts/convert_hf_checkpoint.py --model_size 7B

Note All scripts support argument customization

Once converted, you should have a folder like this:

checkpoints/lit-llama/
├── 7B
│   └── lit-llama.pth
└── tokenizer.model

You are all set. Now you can continue with inference or finetuning.

Try running generate.py to test the imported weights.