InstructTweetSummarizer

This model is a fine-tuned version of facebook/bart-large-cnn on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3548
• Rouge1: 47.5134
• Rouge2: 24.7121
• Rougel: 35.7366
• Rougelsum: 35.6499
• Gen Len: 111.96

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 6
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 12
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	417	0.3468	44.9326	22.3736	33.008	32.9247	116.43
0.5244	2.0	834	0.3440	46.9139	24.683	35.3699	35.333	119.65
0.2061	3.0	1251	0.3548	47.5134	24.7121	35.7366	35.6499	111.96

How to use

Here is how to use this model with the pipeline API:

from transformers import pipeline
summarizer = pipeline("summarization", model="Sidharthkr/InstructTweetSummarizer")
def summarymaker(instruction = "", tweets = ""):
    ARTICLE = f"""[INST] {instruction} [/INST] \\n [TWEETS] {tweets} [/TWEETS]"""
    out = summarizer(ARTICLE, max_length=130, min_length=10, do_sample=False)
    out = out[0]['summary_text'].split("[SUMMARY]")[-1].split("[/")[0].split("[via")[0].strip()
    return out

summarymaker(instruction = "Summarize the tweets for Stellantis in 100 words",
             tweets = """Stellantis - arch critic of Chinese EVs coming to Europe - is in talks with CATL to build a European plant. \n\nIt has concluded that cutting the price of EVs by using Chinese LFP batteries is more important.\n\n@FT story: \nhttps://t.co/l7nGggRFxH. State-of-the-art North America Battery Technology Centre begins to take shape at Stellantis' Automotive Research and Development Centre (ARDC) in Windsor, Ontario.\n\nhttps://t.co/04RO7CL1O5. RT @UAW: 🧵After the historic Stand Up Strike, UAW members at Ford, General Motors and Stellantis have voted to ratify their new contracts,…. RT @atorsoli: Stellantis and CATL are set to supply lower-cost EV batteries together for Europe, signaling automaker's efforts to tighten t…. RT @atorsoli: Stellantis and CATL are set to supply lower-cost EV batteries together for Europe, signaling automaker's efforts to tighten""")
>>> 'Stellantis is in talks with CATL to build a European plant, with a focus on cutting the price of EVs by using Chinese LFP batteries. The company is also developing a state-of-the-art North America Battery Technology Centre in Windsor, Ontario, and has ratified its new contracts with the UAW.'

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0
• Datasets 2.14.7
• Tokenizers 0.14.1