Create README.md

README.md

@@ -0,0 +1,60 @@
+---
+license: mit
+datasets:
+- iamplus/LLama2-SFT-Data
+- iamplus/Open_Platypus_Orca
+- iamplus/Orca
+- iamplus/Conversational_Data
+---
+
+
+**Description :**
+
+This model is trained on a mix of Orca data and Open Source + Closed Multi-turn Conversation data to create a better reasoning model which is capable of holding multi-turn conversations as well.
+
+The Dataset split description, Prompt description as well as Training Parameters are given below.
+
+**Prompt Description :** 
+
+The prompt template for the first turn looks like this:
+```
+<s>[INST] <<SYS>>
+{{ system_prompt }}
+<</SYS>>
+
+{{ user_message }} [/INST]
+```
+
+The prompt template for the multi-turn conversation looks like this:
+```
+<s>[INST] <<SYS>>
+{{ system_prompt }}
+<</SYS>>
+
+{{ user_msg_1 }} [/INST] {{ model_answer_1 }} </s><s>[INST] {{ user_msg_2 }} [/INST]
+```
+
+This model follows the official Meta's chat model Prompt format. Please refer here : https://huggingface.co/blog/llama2#how-to-prompt-llama-2 on how to prompt the model for single/multi-turn conversations.
+
+**Base model :** meta-llama/Llama-2-13b-hf
+
+**Data :** 
+1. 1M Orca dara (Gpt-4 Orca data - OpenOrca)
+2. 1.7M chat data (includes OpenAssistant Chat data, Ultrachat, and many more open source Chat Datasets)
+3. 30k OpenPlatypus data
+
+**Training Params :**
+```
+Number of Epochs : 2
+Batch Size : 128
+Sequence Length : 4096
+Learning Rate : 2e-5 (Cosine)
+Weight Decay : 0.1
+Gradient Clipping : 1.0
+Gamma : 0.85
+beta_1 : 0.9
+beta_2 : 0.95
+eps : 1e-5
+Precision : bf16
+Optimizer : Any Precision AdamW Optimizer
+```