allenai

/

OLMo-7B-hf

like 12

Follow

Ai2 1.78k

Hello @sert121 , OLMo is compatible with VLLM. You can look at this: https://docs.vllm.ai/en/latest/models/supported_models.html.
If you have any particular intermediate checkpoints you are interested in using, then one option is to convert these to HF format yourself (it takes maybe 5-10 mins per checkpoint once you have the checkpoint on your machine). The instructions are in checkpoints.md. The idea is to find the official checkpoint you want in https://github.com/allenai/OLMo/blob/main/checkpoints/official and then use convert_olmo_to_hf_new.py to convert it to HF format.

Hi @amanrangapur - where can I find the weights to manually convert via the script the official checkpoints to the "-HF" variants so I can serve via vLLM? The documentation says "URLs to checkpoints at intermediate steps of our official models' trainings can be found in the csv files under checkpoints/official/. These 'directory' URLs cannot currently be directly accessed, but files within the directory are publicly accessible."

But I'm not sure what the location file format is for a particular model and checkpoint, e.g. Olmo-7B / step150000-tokens664B

Thank you in advance

Hey @aalpat , you can find all the intermediate checkpoints in .csv files: https://github.com/allenai/OLMo/tree/shanea/mup-lr-search/checkpoints/official
The checkpoint folders contain the following files: config.yaml, model.pt, optim.pt, and train.pt.
To access any checkpoint, you can download it by appending the file names at the end of the following URL: https://olmo-checkpoints.org/ai2-llm/olmo-small/w1r5xfzt/step1000-unsharded/model.pt
Let me know if you need further assistance!

Hi @amanrangapur , I can download the URL you shared, but it doesn't seem t owork for this one for example https://olmo-checkpoints.org/ai2-llm/olmo-medium/7gomworq/step150000-unsharded/model.pt which I took from checkpoints in the .csv.

Also a couple of more questions

1. If I understand correctly, I just have to download these files and run convert_olmo_to_hf_new.py on them?
2. In our case, we want to have the add_eos_token param in the tokenizer config, is this something we'd be able to modify?

Hello @aalpat , we have taken down these links since we have released our latest OLMo model which performs better than this version.
Link to latest OLMo release checkpoint files(they are in .csv's): https://github.com/allenai/OLMo/tree/main/configs/official-1124

1. Running convert_olmo_to_hf_new.py: Yes, you’ll need to download the files and run this script to convert the checkpoints to Hugging Face format. Just make sure you’re using the latest release for compatibility(i.e., main branch).
2. Modifying add_eos_token in the tokenizer: Yes, you should be able to modify the tokenizer configuration to include the add_eos_token parameter. You can edit the tokenizer configuration file or modify it programmatically after conversion.