Update README.md
Browse files
README.md
CHANGED
@@ -33,7 +33,18 @@ The project's `initial phase` introduced a knowledge extraction LLM based on LLa
|
|
33 |
- The **full-scale pre-training code** (providing conversion, construction, and loading of large corpora) and **LoRA instruction fine-tuning code** are open-sourced (support multi-machine multi-GPU).
|
34 |
|
35 |
|
36 |
-
All weights have been uploaded to
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
|
38 |
## Contents
|
39 |
|
@@ -606,6 +617,9 @@ Due to time constraints, hardware limitations, and technical reasons, our model
|
|
606 |
|
607 |
- Instruction tuning using full tuning instead of LoRA version is being trained and will be released soon.
|
608 |
- New instruction tuning weights using LoRA will be updated shortly.
|
|
|
|
|
|
|
609 |
- ......
|
610 |
|
611 |
|
@@ -626,7 +640,7 @@ Due to time constraints, hardware limitations, and technical reasons, our model
|
|
626 |
|
627 |
<h2 id="7">7. Others</h2>
|
628 |
|
629 |
-
<h3 id="7-1">7.1 Contributors(
|
630 |
|
631 |
Pretraining:Xiang Chen, Jintian Zhang, Xiaozhuan Liang
|
632 |
|
@@ -638,7 +652,7 @@ Tool learning and Multimodal:Shuofei Qiao, Yixin Ou, Lei Li
|
|
638 |
|
639 |
Model Editing and Safety:Yunzhi Yao, Peng Wang, Siyuan Cheng, Bozhong Tian, Mengru Wang, Zhoubo Li
|
640 |
|
641 |
-
Model Testing and Deployment:Yinuo Jiang, Yuqi Zhu, Hongbin Ye, Zekun Xi
|
642 |
|
643 |
|
644 |
<h3 id="7-2">7.2 Citation</h3>
|
@@ -647,7 +661,7 @@ If you use our repository, please cite the following related papers:
|
|
647 |
|
648 |
```bibtex
|
649 |
@article{cama,
|
650 |
-
author = {Jintian Zhang, Xiaohan Wang, Honghao Gui, Xiang Chen, Yinuo Jiang, Zhen Bi, Jing Chen, Shengyu Mao, Shuofei Qiao, Xiaozhuan Liang, Yixin Ou,
|
651 |
title = {DeepKE-LLM: A Large Language Model Based Knowledge Extraction Toolkit},
|
652 |
year = {2023},
|
653 |
publisher = {GitHub},
|
@@ -657,7 +671,6 @@ If you use our repository, please cite the following related papers:
|
|
657 |
```
|
658 |
|
659 |
|
660 |
-
|
661 |
<h3 id="7-3">7.3 Acknowledgment</h3>
|
662 |
|
663 |
We are very grateful to the following open source projects for their help:
|
|
|
33 |
- The **full-scale pre-training code** (providing conversion, construction, and loading of large corpora) and **LoRA instruction fine-tuning code** are open-sourced (support multi-machine multi-GPU).
|
34 |
|
35 |
|
36 |
+
All weights have been uploaded to HuggingFace🤗. It should be noted that all the following effects are based on `ZhiXi-13B-Diff`. If you have downloaded `ZhiXi-13B-Diff-fp16`, there may be some variations in the effects.
|
37 |
+
| Model Name | Train Method | Weight Type | Size | Download Link | Notes |
|
38 |
+
| -------------- | ------------ | --------------------- | -------- | ---------------------------------- | ------------------------------------------------------------ |
|
39 |
+
| ZhiXi-13B-Diff | Full Pretraining | Differential Weights | 48GB | [HuggingFace](https://huggingface.co/zjunlp/zhixi-13b-diff) <br/> [GoogleDrive](https://drive.google.com/drive/folders/1PZDqZNaBJYQYeON1-9aFBtagktEWAtUK?usp=drive_link)| Restoring the pre-trained weights (i.e. **ZhiXi-13B**) needs to match the weights of `LLaMA-13B`, please refer to [here](#2-2) for specific instructions. |
|
40 |
+
| ZhiXi-13B-Diff-fp16 | Full Pretraining | Differential Weights(fp16) | 24GB | [HuggingFace](https://huggingface.co/zjunlp/zhixi-13b-diff-fp16) <br/> [Google Drive](https://drive.google.com/drive/folders/1LYm-HUSSQ5Rl8nqZcswdiSpcP9xYTXaO?usp=sharing) | The main difference with `ZhiXi-13B-Diff` is the adoption of the `fp16` format for storage, which reduces memory usage. However, it may result in slight differences in the weights obtained from our actual training, which can slightly impact performance. For specific usage instructions, please refer to [here](#2-2) for specific instructions. |
|
41 |
+
| ZhiXi-13B-LoRA | LoRA Instruction-tuning | LoRA Weights | 251MB | [HuggingFace](https://huggingface.co/zjunlp/zhixi-13b-lora) <br/> [GoogleDrive](https://drive.google.com/drive/folders/1GLyaWIyDIayudrQhb_tJYoNPAUk1xByS?usp=drive_link) | It needs to be used with **ZhiXi-13B**. For specific instructions, please refer to [here](#2-4). |
|
42 |
+
| ZhiXi-7B Series | Coming soon | Coming soon | Coming soon | Coming soon | Coming soon |
|
43 |
+
|
44 |
+
|
45 |
+
## NEWS
|
46 |
+
- \[**June 2023**\] The project name has been changed from CaMA to KnowLM.
|
47 |
+
- \[**June 2023**\] Release the first version of pre-trained weights and the LoRA weights.
|
48 |
|
49 |
## Contents
|
50 |
|
|
|
617 |
|
618 |
- Instruction tuning using full tuning instead of LoRA version is being trained and will be released soon.
|
619 |
- New instruction tuning weights using LoRA will be updated shortly.
|
620 |
+
- New models (Llama-7b, Falcon-7b) are being trained (We have limited GPUs!).
|
621 |
+
- New abilities such as molecule and protein generation with [Mol-Instructions](https://github.com/zjunlp/Mol-Instructions), a large-scale biomolecules instruction dataset for large language models.
|
622 |
+
- supporting llama.cpp
|
623 |
- ......
|
624 |
|
625 |
|
|
|
640 |
|
641 |
<h2 id="7">7. Others</h2>
|
642 |
|
643 |
+
<h3 id="7-1">7.1 Contributors(In Random Order)</h3>
|
644 |
|
645 |
Pretraining:Xiang Chen, Jintian Zhang, Xiaozhuan Liang
|
646 |
|
|
|
652 |
|
653 |
Model Editing and Safety:Yunzhi Yao, Peng Wang, Siyuan Cheng, Bozhong Tian, Mengru Wang, Zhoubo Li
|
654 |
|
655 |
+
Model Testing and Deployment:Yinuo Jiang, Yuqi Zhu, Hongbin Ye, Zekun Xi, Xinrong Li
|
656 |
|
657 |
|
658 |
<h3 id="7-2">7.2 Citation</h3>
|
|
|
661 |
|
662 |
```bibtex
|
663 |
@article{cama,
|
664 |
+
author = {Jintian Zhang, Ningyu Zhang, Xiaohan Wang, Honghao Gui, Xiang Chen, Yinuo Jiang, Zhen Bi, Jing Chen, Shengyu Mao, Shuofei Qiao, Xiaozhuan Liang, Yixin Ou, Runnan Fang, Zekun Xi, Xin Xu, Huajun Chen},
|
665 |
title = {DeepKE-LLM: A Large Language Model Based Knowledge Extraction Toolkit},
|
666 |
year = {2023},
|
667 |
publisher = {GitHub},
|
|
|
671 |
```
|
672 |
|
673 |
|
|
|
674 |
<h3 id="7-3">7.3 Acknowledgment</h3>
|
675 |
|
676 |
We are very grateful to the following open source projects for their help:
|