tools/README.md

GPT-NeoX Auxiliary Tools

This directory contains a number of auxiliary tools that are useful for working with GPT-NeoX but not part of the main training code.

Bash

This directory contains some simple, frequently used bash commands to make working on multiple machines easier.

Checkpoints

This directory contains tools for manipulating and converting checkpoints including changing the parallelism settings of a pretrained model, converting between GPT-NeoX and the transformers library, and updating checkpoints trained with Version 1.x of this library to be compatible with Version 2.x.

Datasets

This directory contains tools for downloading and preprocessing datasets to the format expected by the GPT-NeoX library.