abhinavsarkar/Llama-3.1-8b-bnb-4bit-finetuned-Mental-Health-Counselling-Conversations

Model Card for Finetuned Model on Counseling & Support Topics

This model has been fine-tuned on content related to counseling and mental health topics, aimed at providing empathetic responses to user queries on sensitive matters.

Model Details

Model Description

This model is a fine-tuned version of [Meta-Llama-3-8B] on a dataset specific to counseling and mental health [Amod/mental_health_counseling_conversations]. It is designed to handle user queries in a supportive, understanding, and empathetic manner, addressing issues such as mental health, personal growth, and emotional support.

• Developed by: Abhinav Sarkar
• Shared by: abhinavsarkar
• Model type: Causal Language Model
• Languages: English
• Finetuned from model: Meta-Llama-3-8B

Uses

Direct Use

This model can be used for generating responses to queries related to mental health, counseling, and personal support. It provides empathetic, understanding, and supportive responses for users seeking advice or help with personal issues.

Downstream Use

The model can be incorporated into chatbots or virtual assistants aimed at mental health support, initial counseling, and advice applications. Avoid relying on it for critical mental health advice. It should be deployed as a supplementary resource to help users explore initial support topics, with clear disclaimers that it does not substitute professional help.

How to Get Started with the Model

You can use the following pieces of code to get started with the model:

• PreRequisites

!pip install -U bitsandbytes
!pip install -U transformers

• Loading the model and its tokenizer

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained("abhinavsarkar/Llama3-8b-finetuned-Mental-Health-Counseling-Conversations")
tokenizer = AutoTokenizer.from_pretrained("abhinavsarkar/Llama3-8b-finetuned-Mental-Health-Counseling-Conversations")

• Inferencing the model

prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

<|Instruction|>
{}

|<Input|>
{}

<|Response|>
{}
"""

input_text = "I have so many issues to address. I have a history of sexual abuse, I’m a breast cancer survivor and I am a lifetime insomniac. I have a long history of depression and I’m beginning to have anxiety. I have low self esteem but I’ve been happily married for almost 35 years. I’ve never had counseling about any of this. Do I have too many issues to address in counseling?"

inputs = tokenizer([
    prompt.format(
        "You are a Therapist. Respond empathetically and supportively to the user's statement, considering the context of a mental health counseling session", # instruction
        input_text, # input
        "", # output - leave this blank for generation!
    )
], return_tensors="pt").to("cuda" if torch.cuda.is_available() else "cpu")

with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=512)

response = tokenizer.decode(output[0], skip_special_tokens=True)

start_token = "<|Response|>"
end_token = "<|End|>"

start_idx = response.find(start_token) + len(start_token)
end_idx = response.find(end_token)

final_response = response[start_idx:end_idx].strip()
print(final_response)

Possible Errors

If you get an error related to bits and bytes while loading the model, install bitsandbytes and restart your session.

!pip install -U bitsandbytes

After restarting, try loading the model again; it will work fine

If you get an error "Using bitsandbytes 4-bit quantization requires the latest version of bitsandbytes: pip install -U bitsandbytes" try updating transformers

!pip install -U transformers

After restarting, try loading the model again; it will work fine

Training Details

Training Data

The model was fine-tuned on a carefully curated dataset related to mental health, personal support, and counseling topics to provide an empathetic and supportive tone [Amod/mental_health_counseling_conversations].

Training Procedure

The model was fine-tuned using the Hugging Face Transformers library on a cluster of two T4 GPUs. Mixed precision was used for efficient computation, with a batch size of [2], gradient accumulation steps of [4] and a learning rate of [2e-4].

Training Hyperparameters

• Training regime: fp16 mixed precision
• Epochs: 2
• Batch size: 8
• Learning rate: 2e-4

Technical Specifications

Model Architecture and Objective

This model is based on Meta-Llama-3-8B-bnb-4bit, fine-tuned for causal language modeling.

Compute Infrastructure

Hardware

Two T4 GPUs were used in a Kaggle environment for training and fine-tuning.

Software

The model was fine-tuned with PyTorch and Hugging Face’s Transformers library.

More Information

For additional details, please contact https://www.linkedin.com/in/abhinavsarkarrr/.

Model Card Authors

• abhinavsarkar

Model Card Contact

• [email protected]