metadata
license: apache-2.0
datasets:
- HuggingFaceTB/smollm-corpus
- mittagessen/oscar_subset
language:
- en
base_model:
- mittagessen/bytellama_random
This is a ByteLlama 101M model pretrained on the Cosmopedia v2 portion of the SmolLM corpus for 2 epochs, followed by training on a subset of OSCAR for another epoch.