|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
metrics: |
|
- rouge |
|
- bleu |
|
library_name: transformers |
|
--- |
|
# Model Card for Model ID |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1). |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
|
|
|
|
- **Developed by:** **விபின்** |
|
- **Model type:** T5-small |
|
- **Language(s) (NLP):** English |
|
- **License:** Apache 2.0 license |
|
- **Finetuned from model [optional]:** T5-small model |
|
|
|
|
|
## Uses |
|
|
|
This model aims to respond with extractive and abstractive keyphrases for the given content. Kindly use "find keyphrase: " as the task prefix prompt to get the desired outputs. |
|
|
|
|
|
## Bias, Risks, and Limitations |
|
|
|
This model response is based on the inputs given to it. So if any Harmful sentences given to this model, it will respond according to that. |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
``` |
|
from transformers import T5Tokenizer, T5ForConditionalGeneration |
|
import torch |
|
|
|
model_dir = "rv2307/keyphrase-abstraction-t5-small" |
|
tokenizer = T5Tokenizer.from_pretrained(model_dir) |
|
model = T5ForConditionalGeneration.from_pretrained(model_dir, torch_dtype=torch.bfloat16) |
|
device = "cuda" |
|
model.to(device) |
|
|
|
def generate(text): |
|
|
|
text = "find keyphrase: " + text |
|
inputs = tokenizer(text, max_length=512, padding=True, truncation=True, return_tensors='pt') |
|
inputs = {k:v.to(model.device) for k,v in inputs.items()} |
|
|
|
|
|
with torch.no_grad(): |
|
outputs = model.generate( |
|
inputs['input_ids'], |
|
attention_mask=inputs['attention_mask'], |
|
max_length=100, |
|
use_cache=True |
|
) |
|
|
|
output_list = tokenizer.decode(outputs[0],skip_special_tokens=True) |
|
|
|
return output_list |
|
|
|
content = "Hi, How are you??" |
|
outputs = generate(content) |
|
print(outputs) |
|
``` |
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
Mostly used open source datasets for these tasks, which are already available on the huggingface. |
|
|
|
### Training Procedure |
|
|
|
This model has been fine tuned for 6 epochs with 40k datasets collected from the internet. |
|
|
|
|
|
### Results |
|
|
|
``` |
|
Epoch Training Loss Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len |
|
1 0.105800 0.087497 43.840900 19.029900 40.303200 40.320300 16.306200 |
|
2 0.097600 0.081029 46.335000 21.246800 42.377400 42.387500 16.404900 |
|
3 0.091800 0.077546 47.721200 22.467200 43.622400 43.632000 16.308200 |
|
4 0.087600 0.075441 48.633700 23.351300 44.493800 44.504300 16.359000 |
|
5 0.088200 0.074088 48.977500 23.747000 44.804900 44.813200 16.300500 |
|
6 0.084900 0.073381 49.347300 24.029500 45.097100 45.108300 16.332600 |
|
``` |
