Update README.md

README.md

@@ -62,11 +62,7 @@ The following hyperparameters were used during training:
 - rougeLsum: 33.414162%
 
 ### How to use
-Input texts longer than 4000 words can be converted into a list of sub-texts with 4000 words, and pass the list as the input to tokenizer and model.
-Even though the model checkpoint is small, a huge input would crash the memory. Batching the inputs is advised. 
-Rule of thumb is that T4 can handle at a time a list of upto 14-15 elements, with each elements having 4000 words.
-
-Note 'max_new_tokens=60' is used in the example below to limit the summary size. BART model has max generation length = 142 (default) and min generation length = 56.
+Note 'max_new_tokens=60' is used in the example below to control the summary size. BART model has max generation length = 142 (default) and min generation length = 56.
 
 ```python
 import torch
@@ -85,9 +81,10 @@ tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
 model = PeftModel.from_pretrained(model, peft_model_id, device_map='auto')
 
 # Tokenize the text inputs
-texts = "<e.g. Part of YouTube Transcript>"
-inputs = tokenizer(texts, return_tensors="pt", padding=True, truncation=True)
+texts = "<e.g. Transcript>"
+inputs = tokenizer(texts, return_tensors="pt", padding=True, ) # truncation=True
 
+# Make inferences
 device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
 with torch.no_grad():    
     output = self.model.generate(input_ids=inputs["input_ids"].to(device), max_new_tokens=60, do_sample=True, top_p=0.9)