ayoolaolafenwa
commited on
Commit
•
6dbd6e9
1
Parent(s):
3dfe921
Update README.md
Browse files
README.md
CHANGED
@@ -7,7 +7,7 @@ pipeline_tag: conversational
|
|
7 |
---
|
8 |
|
9 |
## ChatLM
|
10 |
-
It is a chat Large Language
|
11 |
and trained on [chat-bot-instructions prompts dataset](https://huggingface.co/datasets/ayoolaolafenwa/sft-data).
|
12 |
ChatLM was trained on a dataset containing normal day to day human conversations, due to limited data used in training
|
13 |
it does not generalize well for tasks like coding and current affairs.
|
@@ -123,12 +123,12 @@ new_data = pd.DataFrame({"prompt": prompts, "response": responses})
|
|
123 |
# Write the new dataframe to a csv file
|
124 |
new_data.to_csv("MyData/chatbot_instruction_prompts_train.csv", index=False)
|
125 |
```
|
126 |
-
The
|
127 |
Check the the modified dataset https://huggingface.co/datasets/ayoolaolafenwa/sft-data .
|
128 |
|
129 |
### Training
|
130 |
|
131 |
ChatLM was supervised finetuned with pretrained [Falcon 1-Billion parameters model](https://huggingface.co/tiiuae/falcon-rw-1b) trained on 350-Billion tokens
|
132 |
-
of RefinedWeb. It was trained with a single H100 GPU for 1 epoch. Check the full code for supervised finetune
|
133 |
training on its github repository https://github.com/ayoolaolafenwa/ChatLM/tree/main
|
134 |
|
|
|
7 |
---
|
8 |
|
9 |
## ChatLM
|
10 |
+
It is a chat Large Language Model finetuned with pretrained [Falcon-1B model](https://huggingface.co/tiiuae/falcon-rw-1b)
|
11 |
and trained on [chat-bot-instructions prompts dataset](https://huggingface.co/datasets/ayoolaolafenwa/sft-data).
|
12 |
ChatLM was trained on a dataset containing normal day to day human conversations, due to limited data used in training
|
13 |
it does not generalize well for tasks like coding and current affairs.
|
|
|
123 |
# Write the new dataframe to a csv file
|
124 |
new_data.to_csv("MyData/chatbot_instruction_prompts_train.csv", index=False)
|
125 |
```
|
126 |
+
The users` prompts in the dataset are appended with the tag <user> and the corresponding responses with the tag <chatbot>.
|
127 |
Check the the modified dataset https://huggingface.co/datasets/ayoolaolafenwa/sft-data .
|
128 |
|
129 |
### Training
|
130 |
|
131 |
ChatLM was supervised finetuned with pretrained [Falcon 1-Billion parameters model](https://huggingface.co/tiiuae/falcon-rw-1b) trained on 350-Billion tokens
|
132 |
+
of RefinedWeb. It was trained with a single H100 GPU for 1 epoch. It achieves Perplexity *1.738*. Check the full code for supervised finetune
|
133 |
training on its github repository https://github.com/ayoolaolafenwa/ChatLM/tree/main
|
134 |
|