metadata
base_model: JackFram/llama-68m
language:
- en
license: apache-2.0
library_name: transformers
datasets:
- allenai/c4
Model Description
This model is based of llama-68m and was initialized from this model with tied embeddings and then further pretrained on the uncleaned c4 dataset. This model is obviously a beta based on the name and has not been trained on the full training set which is the first 1 million samples of the uncleaned c4 dataset as mentioned above.