Update README.md

README.md

@@ -1,201 +1,214 @@
 ---
 library_name: transformers
 tags:
-- trl
-- sft
+- SFT
+- PEFT
+- Mental Health
+- Psychotherapy
+- Fine-tuning
+- Text Generation
+- Chatbo
+license: mit
+datasets:
+- mpingale/mental-health-chat-dataset
+- Amod/mental_health_counseling_conversations
+- heliosbrahma/mental_health_chatbot_dataset
 ---
 
 # Model Card for Model ID
 
 <!-- Provide a quick summary of what the model is/does. -->
+A fine-tuned Llama-3-8B-instruct trained to provide initial support and guidance in mental health by leveraging the power of large language models (LLMs).
+It's a fine-tuned version of the LLaMA 3 model specifically trained on mental health counseling conversations.
 
+![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/65f2cc4457fbd3c9fb65b34c/ujViqdS9zgW-IrEFbAKLw.jpeg)
 
 
 ## Model Details
 
-### Model Description
+Llama-3-8B-chat-psychotherapist
 
-<!-- Provide a longer summary of what this model is. -->
+Provides active listening: The model is trained to follow conversation flows and respond attentively to user expressions.
+Offers empathetic support: The model's responses are designed to be supportive and understanding, fostering a safe space for users to share their concerns.
+Guides users towards self-reflection: The model can ask open-ended questions and prompt users to explore their thoughts and feelings.
+Offers resources and information: Depending on the conversation, the model can provide users with relevant mental health resources and information.
+Important to Note:
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+**This model is intended for initial support and guidance, not a replacement for professional mental health care.
+It's crucial to emphasize that users should seek professional help if needed.
+The model is still under development, and its responses may require human oversight for accuracy and safety.**
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+Further Considerations:
+You can specify the type of mental health conversations the model is trained on (e.g., anxiety, depression, general well-being).
+Briefly mention the dataset used for fine-tuning to showcase the model's training grounds.
+If applicable, mention any limitations of the model, such as inability to diagnose or provide specific treatment plans.
 
-### Model Sources [optional]
 
-<!-- Provide the basic links for the model. -->
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
+## How to Get Started with the Model
+Use the code below to get started with the model.
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+```
 
-[More Information Needed]
+from transformers import AutoTokenizer, AutoModel
 
-### Out-of-Scope Use
+tokenizer = AutoTokenizer.from_pretrained("zementalist/llama-3-8B-chat-psychotherapist")
+model = AutoModel.from_pretrained("zementalist/llama-3-8B-chat-psychotherapist")
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+dialogue = "I feel like I don't exist and my body is not my own, I'm sombody else observing me, what causes these ideas?"
 
-[More Information Needed]
+messages = [
+ {"role": "system", "content": "Reply to the following inquiry:"},
+ {"role": "user", "content": dialogue}
+]
 
-## Bias, Risks, and Limitations
+input_ids = tokenizer.apply_chat_template(
+ messages,
+ add_generation_prompt=True,
+ return_tensors="pt"
+).to(model.device)
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
 
-[More Information Needed]
+terminators = [
+ tokenizer.eos_token_id,
+ tokenizer.convert_tokens_to_ids("<|eot_id|>")
+]
 
-### Recommendations
+outputs = model.generate(
+ input_ids,
+ max_new_tokens=256,
+ eos_token_id=terminators,
+ do_sample=True,
+ temperature=0.01
+)
+response = outputs[0][input_ids.shape[-1]:]
+output = tokenizer.decode(response, skip_special_tokens=True)
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+print(output)
 
-## How to Get Started with the Model
+```
 
-Use the code below to get started with the model.
 
-[More Information Needed]
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
+1. [Amod/mental_health_counseling_conversations](https://huggingface.co/datasets/Amod/mental_health_counseling_conversations)
+2. [mpingale/mental-health-chat-dataset](https://huggingface.co/datasets/mpingale/mental-health-chat-dataset)
+3. [heliosbrahma/mental_health_chatbot_dataset](https://huggingface.co/datasets/heliosbrahma/mental_health_chatbot_dataset)
 
 
 #### Training Hyperparameters
 
 - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
 
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
+* epochs: 2
+* learning rate: 6e-5
+* learning rate scheduler type: constant
+* Warmup ratio: 0.03
+* optimizer: paged_adamw_32bit
+* weight_decay: 0.001
+* fp16: True
 
 ## Evaluation
 
 <!-- This section describes the evaluation protocols and provides the results. -->
 
-### Testing Data, Factors & Metrics
-
-#### Testing Data
 
-<!-- This should link to a Dataset Card if possible. -->
 
-[More Information Needed]
+| **Step** | **Training Loss** |
+|---------|-------------------|
+| 100 | 2.637700 |
+| 200 | 2.350900 |
+| 300 | 2.234500 |
+| 400 | 2.230800 |
+| 500 | 2.194900 |
+| 600 | 2.041100 |
+| 700 | 2.033800 |
+| 800 | 1.988600 |
+| 900 | 1.898900 |
+| 1000 | 1.823600 |
+| 1100 | 1.943700 |
+| 1200 | 1.789900 |
+| 1300 | 1.840700 |
+| 1400 | 1.760900 |
+| 1500 | 1.718400 |
+| 1600 | 1.725600 |
+| 1700 | 1.717600 |
+| 1800 | 1.628200 |
+| 1900 | 1.693300 |
+| 2000 | 1.535600 |
+| 2100 | 1.630100 |
+| 2200 | 1.592300 |
+| 2300 | 1.628400 |
+| 2400 | 1.462500 |
+| 2500 | 1.504500 |
+| 2600 | 1.517500 |
+| 2700 | 1.375100 |
+| 2800 | 1.452800 |
+| 2900 | 1.527200 |
+| 3000 | 1.418600 |
+| 3100 | 1.437700 |
+| 3200 | 1.247900 |
+| 3300 | 1.134900 |
+| 3400 | 1.283800 |
+| 3500 | 1.378600 |
+| 3600 | 1.114600 |
+| 3700 | 1.300900 |
+| 3800 | 1.225700 |
+| 3900 | 1.190000 |
+| 4000 | 1.260200 |
+| 4100 | 1.053300 |
+| 4200 | 1.173000 |
+| 4300 | 1.275900 |
+| 4400 | 1.079800 |
+| 4500 | 1.121100 |
+| 4600 | 1.043100 |
+| 4700 | 1.121000 |
+| 4800 | 1.087200 |
+| 4900 | 1.102400 |
+| 5000 | 1.126100 |
+| 5100 | 1.182200 |
+| 5200 | 1.156500 |
+| 5300 | 0.979700 |
+| 5400 | 1.145300 |
+| 5500 | 1.018100 |
+| 5600 | 1.070500 |
+| 5700 | 0.954900 |
+| 5800 | 1.074200 |
+| 5900 | 1.022800 |
+| 6000 | 1.037700 |
+| 6100 | 0.913400 |
+| 6200 | 0.929500 |
 
-#### Factors
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
 
 ### Results
 
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
+1. Question: I feel like I don't exist and my body is not my own,
+like if I'm sombody else observin me, what could be this disorder?
+* * Answer: You're describing a phenomenon called depersonalization. It's a common symptom of anxiety disorders, post-traumatic stress disorder (PTSD), and other mental health conditions.
 
-## Glossary [optional]
+Depersonalization can make you feel like you're observing yourself from the outside, like you're watching a movie or reading a book about yourself. You might also feel like you're not in control of your body or that you're just a puppet.
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+This feeling can be very distressing and can make you wonder if you're even alive. It's a very disorienting and unsettling experience.
 
-[More Information Needed]
+If you're experiencing depersonalization, it's important to seek help from a mental health professional. They can help you identify the underlying causes of your feelings and develop coping strategies to manage them.
 
-## More Information [optional]
 
-[More Information Needed]
+2. Question: I'm always confused when making decisions, I cannot choose one option,
+I must have only one option so I can make a decision, what are the causes of being
+indecisive?
+* * Answer: Indecisiveness is a common phenomenon that many people experience. It's not a mental health condition, but it can be frustrating and time-consuming. Here are some possible causes of indecisiveness:
 
-## Model Card Authors [optional]
+1. Fear of making the wrong decision: Sometimes, people are afraid of making the wrong choice, which can lead to indecisiveness. This fear can be due to various factors such as fear of failure, fear of rejection, or fear of the unknown.
+2. Lack of clear goals: When people don't have clear goals, they can be indecisive. This is because they don't know what they want, which can lead to confusion and indecisiveness.
+3. Overthinking: Overthinking can be a major cause of indecisiveness. When people overthink, they can get caught up in their thoughts and be unable to make a decision.
+4. Lack of self-confidence: People who lack self-confidence can be indecisive. This is because they don't believe in themselves and are afraid of making the wrong decision.
+5. Avoidance: Avoiding a decision can be a cause of indecisiveness. When people avoid making a decision, they can be indecisive because they don't know
 
-[More Information Needed]
 
-## Model Card Contact
 
-[More Information Needed]
+- **Developed by:** [Zeyad Khalid]