# Fine-Tuned GPT-2 Model for Sentiment Analysis

### Model Description

<!--  -->
This fine-tuned GPT-2 model is designed for sentiment analysis of tweets. 
Trained on the mteb/tweet_sentiment_extraction dataset, the model classifies tweets into three sentiment categories Positive, Neutral, and Negative.


- **Developed by:** Pranav Sethi
- **Model type:** gpt2
- **Language(s) (NLP):** Large Language Model
- **License:** Apachee License 2.0


## Uses

This model is intended for sentiment analysis of tweets or other short social media texts. It is designed for researchers, developers,
and data scientists who want to classify textual data into three sentiment categories: Positive, Neutral, or Negative.
End users analyzing sentiment trends based on this model's predictions


### Training Data

The model is fine-tuned on the Tweet Sentiment Extraction dataset

- Dataset: mteb/tweet_sentiment_extraction



### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

This model was fine-tuned using the Hugging Face Transformers library with the following training configuration:

#### Training Hyperparameters

- Batch size: 1
- Training epochs: 3
- Base model: GPT-2
- gradient_accumulation_steps: 4



## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

The testing was performed on the Tweet Sentiment Extraction dataset


#### Metrics

- **Base Model**: GPT-2
- **Task**: Sentiment Classification (Positive, Neutral, Negative)
- **Dataset**: Tweet Sentiment Extraction
- **Evaluation Metric**: Accuracy



### Results

The model achieved an accuracy of 73% on the test dataset.

## Usage
This model is designed for sentiment analysis of tweets or other short social media text. Given an input text, it predicts the sentiment as Positive, Neutral, or Negative.



### Example Code
```python
from datasets import load_dataset
import pandas as pd
from transformers import GPT2ForSequenceClassification
from transformers import TrainingArguments, Trainer
import evaluate
import torch

# Load the model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("pranavsethi01/yps1382-allin")
model = AutoModelForSequenceClassification.from_pretrained("pranavsethi01/yps1382-allin")

# Perform sentiment analysis
inputs = tokenizer("I love Hugging Face!", return_tensors="pt")
outputs = model(**inputs)
predictions = torch.argmax(outputs.logits, dim=-1)

# Output sentiment (0: Negative, 1: Neutral, 2: Positive)
print("Sentiment:", predictions.item())

# Limitations
- ** Bias **: The dataset may contain biased or harmful text, potentially influencing predictions.
- ** Domain Limitations **: Optimized for English tweets; performance may degrade on other text types or languages.

# Ethical Considerations
This model should be used responsibly. Be aware of biases in the training data and avoid deploying the model in sensitive or high-stakes applications without further validation.

# Acknowledgments
- Hugging Face Transformers library
- mteb/tweet_sentiment_extraction dataset