# Better Language Models and Their Implications (GPT2)
<b>Paper:</b> <a href="https://openai.com/blog/better-language-models/">https://openai.com/blog/better-language-models/</a>  
 
# About
These are the pretrained weights for [this](https://github.com/matthias-wright/flaxmodels/tree/main/flaxmodels/gpt2) ResNet implementation in Jax/Flax. The weights are taken from [this](https://github.com/huggingface/transformers/tree/master/src/transformers/models/gpt2) repository. 

# Documentation
[Here](https://github.com/matthias-wright/flaxmodels/blob/main/docs/Documentation.md#1-checkpoints) is a documentation that explains the preprocessing steps as well as the format of the pretrained weights.