---
license: apache-2.0
language:
- en
metrics:
- rouge
- bleu
library_name: transformers
---
# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** **விபின்**
- **Model type:** T5-small 
- **Language(s) (NLP):** English
- **License:** Apache 2.0 license
- **Finetuned from model [optional]:** T5-small model


## Uses

This model aims to respond with extractive and abstractive keyphrases for the given content. Kindly use "find keyphrase: " as the task prefix prompt to get the desired outputs.


## Bias, Risks, and Limitations

This model response is based on the inputs given to it. So if any Harmful sentences given to this model, it will respond according to that.


## How to Get Started with the Model

```
from transformers import T5Tokenizer, T5ForConditionalGeneration
import torch

model_dir = "rv2307/keyphrase-abstraction-t5-small"
tokenizer = T5Tokenizer.from_pretrained(model_dir)
model = T5ForConditionalGeneration.from_pretrained(model_dir, torch_dtype=torch.bfloat16)
device = "cuda"
model.to(device)

def generate(text):
    
    text = "find keyphrase: "  + text
    inputs = tokenizer(text, max_length=512, padding=True, truncation=True, return_tensors='pt')
    inputs = {k:v.to(model.device) for k,v in inputs.items()}

    
    with torch.no_grad():
        outputs = model.generate(
            inputs['input_ids'],
            attention_mask=inputs['attention_mask'],
            max_length=100,
            use_cache=True
        )

    output_list = tokenizer.decode(outputs[0],skip_special_tokens=True)
        
    return output_list

content = "Hi, How are you??"
outputs = generate(content)
print(outputs)
```

## Training Details

### Training Data

Mostly used open source datasets for these tasks, which are already available on the huggingface.

### Training Procedure

This model has been fine tuned for 6 epochs with 40k datasets collected from the internet. 


### Results

```
Epoch	Training Loss	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1	0.105800	0.087497	43.840900	19.029900	40.303200	40.320300	16.306200
2	0.097600	0.081029	46.335000	21.246800	42.377400	42.387500	16.404900
3	0.091800	0.077546	47.721200	22.467200	43.622400	43.632000	16.308200
4	0.087600	0.075441	48.633700	23.351300	44.493800	44.504300	16.359000
5	0.088200	0.074088	48.977500	23.747000	44.804900	44.813200	16.300500
6	0.084900	0.073381	49.347300	24.029500	45.097100	45.108300	16.332600
```