| In its current state, this model has not yet been finetuned on long-context data. Do not use this model without performing additional pretraining, as performance will likely suffer. | |
| Similar to the other publicly available LSG models, this model was created using the LSG conversion script found here: | |
| https://github.com/ccdv-ai/convert_checkpoint_to_lsg |