license: apache-2.0
datasets:
- stanfordnlp/imdb
- bharatiyabytes/sentimentWithSarcasm
language:
- en
base_model:
- google/flan-t5-small
pipeline_tag: text-classification
tags:
- sarcasm
- gen-ai
- llms
Model Card for Sarcasm-Enhanced Sentiment Analysis Model
This model performs sentiment analysis with a specific focus on detecting sarcasm in textual content. It is fine-tuned on a combination of standard sentiment datasets and specialized sarcastic data, allowing for more nuanced sentiment classification that accounts for sarcastic language.
Model Details
Model Description
This model leverages the Flan-T5-small transformer architecture, fine-tuned on datasets including IMDB for sentiment analysis and bharatiyabytes/sentimentWithSarcasm for sarcasm detection. By combining these datasets, the model is better equipped to differentiate between sarcastic and genuine sentiment expressions, improving sentiment analysis accuracy in contexts where sarcasm is prevalent.
- Developed by: bharatiyabytes
- Funded by [optional]: Not yet funded
- Model type: Text Classification
- Language(s) (NLP): English (en)
- License: Apache 2.0
- Fine-tuned from model: google/flan-t5-small
Model Sources [optional]
- Repository: https://github.com/sohi-g/lets-talk-the-hype.git
- Paper [optional]: [Link to paper if available]
- Demo [optional]: [Link to demo if available]
Uses
Direct Use
This model can be used directly for sentiment classification on texts where sarcasm may obscure intended sentiment. It provides accurate classifications by considering nuanced expressions, making it ideal for social media analysis, customer feedback processing, and sarcasm-rich content sources.
Downstream Use
This model can serve as a base for further fine-tuning for domains that require sarcasm-aware sentiment analysis, such as customer service, public relations, and social media monitoring applications.
Out-of-Scope Use
The model may not perform well on texts with heavy dialects or informal language that goes beyond the sarcasm in the fine-tuning data. It is not intended for multi-lingual sarcasm detection.
Bias, Risks, and Limitations
As with many sentiment analysis models, there may be inherent biases in the sarcasm and sentiment labels in the training datasets, potentially affecting model performance across different demographic or cultural groups. Users should be cautious when using this model in critical decision-making contexts.
Recommendations
Users should perform additional validation on specific datasets to ensure that model predictions align with intended use cases, especially in high-stakes applications.
How to Get Started with the Model
Use the code below to get started with the model:
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tokenizer = AutoTokenizer.from_pretrained("your-username/sarcasm-sentiment-analysis")
model = AutoModelForSequenceClassification.from_pretrained("your-username/sarcasm-sentiment-analysis")
inputs = tokenizer("Your input text here", return_tensors="pt")
outputs = model(**inputs)
predictions = torch.argmax(outputs.logits, dim=-1)
Training Details
Training Data
The model was fine-tuned on a combination of IMDB (a general sentiment analysis dataset) and bharatiyabytes/sentimentWithSarcasm (designed to capture sarcastic sentiment). This blend improves the model’s ability to identify nuanced sentiment.
Training Procedure
Preprocessing
Standard text preprocessing methods were applied, such as tokenization and lowercase transformation.
Training Hyperparameters
Training regime: FP32 Epochs: 3 Batch Size: 16 Learning Rate: 3e-5
Speeds, Sizes, Times
Training took approximately 4 hours on an NVIDIA V100 GPU with a model size of 60M parameters.
Evaluation
Testing Data, Factors & Metrics
Testing Data
he model was evaluated on the IMDB and bharatiyabytes/sentimentWithSarcasm test splits.
Factors
Evaluation considers sentiment disaggregation by sarcastic vs. non-sarcastic samples.
Metrics
-Accuracy: Measures overall sentiment classification accuracy. -F1 Score (Sarcasm): Evaluates the model’s sarcasm detection capability, which is key for accurate sarcastic sentiment handling.
Results
The model achieved an accuracy of 88% on the sentiment classification task and an F1 score of 0.83 on sarcasm detection.
Summary
The model shows strong performance in sarcasm-sensitive sentiment analysis, making it suitable for applications where nuanced sentiment interpretation is crucial.
Model Examination
The model’s predictions have been examined to ensure that sarcastic content is accurately labeled, using interpretability tools such as SHAP to visualize model attention on sarcastic phrases.
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: NVIDIA V100 GPU
- Hours used: Approximately 4 hours
- Cloud Provider: Google
- Compute Region: USA
- Carbon Emitted: NA
Technical Specifications
Model Architecture and Objective
The model uses the Flan-T5-small architecture fine-tuned for binary sentiment classification with sarcasm detection as an enhancement.
Compute Infrastructure
[More Information Needed]
Hardware
NVIDIA V100 GPU
Software
Hugging Face Transformers Library, PyTorch
Citation [optional]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors
- Shivang sinha
- Garima Sohi
- Parteek