Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
|
@@ -35,7 +35,7 @@ Denomination specific models like ChristianGPT-catholic are based on ChristianGP
|
|
| 35 |
- GPU with over 16GB of memory (we trained on NVIDIA Tesla V100 32GB and NVIDIA Ampere A6000 45GB)
|
| 36 |
- 30GB of RAM (the raw model weights are about 29GB, our models are cast to 8bit to use less memory)
|
| 37 |
|
| 38 |
-
|
| 39 |
We leaned heavily on open-source libraries like `transformers`, `peft`, and `bitsandbytes` for this project.
|
| 40 |
- Dettmers, Tim, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. "[LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://arxiv.org/abs/2208.07339)." *arXiv preprint arXiv:2208.07339*.
|
| 41 |
- Hu, Edward J., Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. "[LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)." *arXiv preprint arXiv:2106.09685*.
|
|
|
|
| 35 |
- GPU with over 16GB of memory (we trained on NVIDIA Tesla V100 32GB and NVIDIA Ampere A6000 45GB)
|
| 36 |
- 30GB of RAM (the raw model weights are about 29GB, our models are cast to 8bit to use less memory)
|
| 37 |
|
| 38 |
+
## References
|
| 39 |
We leaned heavily on open-source libraries like `transformers`, `peft`, and `bitsandbytes` for this project.
|
| 40 |
- Dettmers, Tim, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. "[LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://arxiv.org/abs/2208.07339)." *arXiv preprint arXiv:2208.07339*.
|
| 41 |
- Hu, Edward J., Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. "[LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)." *arXiv preprint arXiv:2106.09685*.
|