Update README.md

README.md

@@ -14,16 +14,55 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-# spam_not_spam
+# RoBERTa-PEFT-ForSequenceClassification
 
-This model is a fine-tuned version of [FacebookAI/roberta-base](https://huggingface.co/FacebookAI/roberta-base) on an unknown dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.0414
-- Accuracy: 0.9839
+This model is a fine-tuned version of [FacebookAI/roberta-base](https://huggingface.co/FacebookAI/roberta-base) on `spam_not_spam` dataset.
+It achieves the following results on the evaluation set: \
+*- Loss: 0.0414* \
+*- Accuracy: 0.9839*
 
 ## Model description
 
-More information needed
+### Performing Parameter-Efficient Fine-Tuning 
+We used low rank adaptation (LoRA) from PEFT library in HuggineFace. \
+Base-model is finetuned using LoRA config below: \
+`peft_config = LoraConfig(task_type=TaskType.SEQ_CLS, inference_mode=False, r=8, lora_alpha=32, lora_dropout=0.1)`
+
+### Training
+Use script below for model fine-tuning:
+
+```
+def compute_metrics(eval_pred):
+    predictions, labels = eval_pred
+    predictions = np.argmax(predictions, axis=1)
+    return {"accuracy": (predictions == labels).mean()}
+
+trainer = Trainer(
+    model=lora_model,
+    args=TrainingArguments(
+        output_dir="./data/spam_not_spam",
+        # Set the learning rate
+        learning_rate = 2e-5,
+        # Set the per device train batch size and eval batch size
+        per_device_train_batch_size=16,
+        per_device_eval_batch_size=64,
+        # Evaluate and save the model after each epoch
+        evaluation_strategy = "epoch",
+        save_strategy = "epoch",
+        num_train_epochs=5,
+        weight_decay=0.01,
+        load_best_model_at_end=True,
+    ),
+    train_dataset= tokenized_dataset_train,
+    eval_dataset= tokenized_dataset_test,
+    tokenizer=tokenizer,
+    data_collator=DataCollatorWithPadding(tokenizer=tokenizer),
+    compute_metrics=compute_metrics,
+)
+```
+
+
+
 
 ## Intended uses & limitations