ltg/gpt-bert-babylm-base

Submission to the BabyLM challenge 2024 trained on Baby-cosmo-fine-100M.

The training scripts are published here: https://github.com/ltgoslo/gpt-bert

@misc{charpentier2024gptbertboth,
      title={GPT or BERT: why not both?}, 
      author={Lucas Georges Gabriel Charpentier and David Samuel},
      year={2024},
      eprint={2410.24159},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2410.24159}, 
}