File size: 1,373 Bytes
4a26537 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
---
# Liger-GLA-8B
[\[📂 GitHub\]](https://github.com/OpenSparseLLMs/Linearization) [\[📜 Liger\]](https://huggingface.co/papers/2503.01496) [\[📑 arXiv\]](https://arxiv.org/abs/2503.01496)
We introduce **Liger-GSA-8B**, a gated linear recurrent model linearized from Transformer-based LLM.
<p align="center">
<img width="90%" src="figures/liger_framework.png">
</p>
Our **Liger** framework is compatible with various linear recurrent models with gating structures:
| Model Name | Base Model | Linear Structure | HF Link |
| --- | --- | --- | --- |
| Liger-GLA-8B | [Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) | [GLA](https://arxiv.org/abs/2312.06635) | [🤗 link](https://huggingface.co/linear-moe-hub/Liger-GLA-8B) |
| Liger-GSA-8B | [Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) | [GSA](https://arxiv.org/abs/2409.07146) | [🤗 link](https://huggingface.co/linear-moe-hub/Liger-GSA-8B) |
## Citation
If you find this repo useful, please cite and star our work:
```bibtex
@article{lan2025liger,
title={Liger: Linearizing Large Language Models to Gated Recurrent Structures},
author={Lan, Disen and Sun, Weigao and Hu, Jiaxi and Du, Jusen and Cheng, Yu},
journal={arXiv preprint arXiv:2503.01496},
year={2025}
}
``` |