---
library_name: peft
license: apache-2.0
pipeline_tag: text-classification
tags:
- hatespeech
- hatecot
- cot
- llama
---

## Introduction
This is the LoRA-adapater for the Llama-7B introduced in the paper 
*HateCOT: An Explanation-Enhanced Dataset for Generalizable Offensive Speech Detection via Large Language Models*.
The base model is instruction-finetuned on 52,000 samples that includes augmented humman annotation to produce 
legible explanations based on predefined criteria in the **provided definition**.


To use the model, please load along with the original Llama model (detailed configuration in the *Training Procedure*). 
For instruction to load Peft models: https://huggingface.co/docs/transformers/main/en/peft

These adapters can also be finetuned on a new set of data. See the article for more details.

## Usage
Use the following template to prompt the model:
```
### Instruction
Perform this task by considering the following Definitions.
Based on the message, label the input as only one of the following categories:
[Class 1], [Class 2], ..., or [Class N].
Provide a brief paragraph to explain step-by-step why the post should be classsified
with the provided Label based on the given Definitions. If this post targets a group or
entity relevant to the definition of the specified Label, explain who this target is and how
that leads to that Label.
Append the string '<END>' to the end of your response. Provide your response in the following format:
EXPLANATION: [text]
LABEL:[text] <END>
### Definitions:
[Class 1]: [Definition 1]
[Class 2]: [Definition 2]
...
[Class N]: [Definition 3]
### Input
{post}
### Response:
```

## Citation
```bibtex
@article{nghiem2024hatecot,
  title={HateCOT: An Explanation-Enhanced Dataset for Generalizable Offensive Speech Detection via Large Language Models},
  author={Nghiem, Huy and Daum{\'e} III, Hal},
 journal={arXiv preprint arXiv:2403.11456},
  year={2024}
}
```

## Original Model
Please visit the main repository to gain permission to download original model weights.

https://huggingface.co/meta-llama



## Training procedure


The following `bitsandbytes` quantization config was used during training:
- quant_method: bitsandbytes
- load_in_8bit: True
- load_in_4bit: False
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: False
- bnb_4bit_compute_dtype: float16
### Framework versions


- PEFT 0.5.0