File size: 2,809 Bytes
b7ae4f8 16b03fe b7ae4f8 16b03fe ab72170 16b03fe ab72170 16b03fe ab72170 16b03fe ab72170 16b03fe ab72170 16b03fe ab72170 16b03fe ab72170 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 |
---
license: apache-2.0
language:
- en
metrics:
- rouge
- bleu
library_name: transformers
---
# Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->
This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
- **Developed by:** **விபின்**
- **Model type:** T5-small
- **Language(s) (NLP):** English
- **License:** Apache 2.0 license
- **Finetuned from model [optional]:** T5-small model
## Uses
This model aims to respond with extractive and abstractive keyphrases for the given content. Kindly use "find keyphrase: " as the task prefix prompt to get the desired outputs.
## Bias, Risks, and Limitations
This model response is based on the inputs given to it. So if any Harmful sentences given to this model, it will respond according to that.
## How to Get Started with the Model
```
from transformers import T5Tokenizer, T5ForConditionalGeneration
import torch
model_dir = "rv2307/keyphrase-abstraction-t5-small"
tokenizer = T5Tokenizer.from_pretrained(model_dir)
model = T5ForConditionalGeneration.from_pretrained(model_dir, torch_dtype=torch.bfloat16)
device = "cuda"
model.to(device)
def generate(text):
text = "find keyphrase: " + text
inputs = tokenizer(text, max_length=512, padding=True, truncation=True, return_tensors='pt')
inputs = {k:v.to(model.device) for k,v in inputs.items()}
with torch.no_grad():
outputs = model.generate(
inputs['input_ids'],
attention_mask=inputs['attention_mask'],
max_length=100,
use_cache=True
)
output_list = tokenizer.decode(outputs[0],skip_special_tokens=True)
return output_list
content = "Hi, How are you??"
outputs = generate(content)
print(outputs)
```
## Training Details
### Training Data
Mostly used open source datasets for these tasks, which are already available on the huggingface.
### Training Procedure
This model has been fine tuned for 6 epochs with 40k datasets collected from the internet.
### Results
```
Epoch Training Loss Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
1 0.105800 0.087497 43.840900 19.029900 40.303200 40.320300 16.306200
2 0.097600 0.081029 46.335000 21.246800 42.377400 42.387500 16.404900
3 0.091800 0.077546 47.721200 22.467200 43.622400 43.632000 16.308200
4 0.087600 0.075441 48.633700 23.351300 44.493800 44.504300 16.359000
5 0.088200 0.074088 48.977500 23.747000 44.804900 44.813200 16.300500
6 0.084900 0.073381 49.347300 24.029500 45.097100 45.108300 16.332600
``` |