tr416 commited on
Commit
a05b67a
1 Parent(s): 9e25262

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -1
README.md CHANGED
@@ -33,4 +33,9 @@ Denomination specific models like ChristianGPT-catholic are based on ChristianGP
33
 
34
  ## Training requirements:
35
  - GPU with over 16GB of memory (we trained on NVIDIA Tesla V100 32GB and NVIDIA Ampere A6000 45GB)
36
- - 30GB of RAM (the raw model weights are about 29GB, our models are cast to 8bit to use less memory)
 
 
 
 
 
 
33
 
34
  ## Training requirements:
35
  - GPU with over 16GB of memory (we trained on NVIDIA Tesla V100 32GB and NVIDIA Ampere A6000 45GB)
36
+ - 30GB of RAM (the raw model weights are about 29GB, our models are cast to 8bit to use less memory)
37
+
38
+ - ## References
39
+ We leaned heavily on open-source libraries like `transformers`, `peft`, and `bitsandbytes` for this project.
40
+ - Dettmers, Tim, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. "[LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://arxiv.org/abs/2208.07339)." *arXiv preprint arXiv:2208.07339*.
41
+ - Hu, Edward J., Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. "[LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)." *arXiv preprint arXiv:2106.09685*.