Model save

README.md

@@ -0,0 +1,79 @@
+---
+base_model: google/pegasus-xsum
+tags:
+- generated_from_trainer
+metrics:
+- rouge
+- precision
+- recall
+- f1
+model-index:
+- name: LLM_Teached_Pegasus_100k
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# LLM_Teached_Pegasus_100k
+
+This model is a fine-tuned version of [google/pegasus-xsum](https://huggingface.co/google/pegasus-xsum) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.5409
+- Rouge1: 0.4869
+- Rouge2: 0.2373
+- Rougel: 0.406
+- Rougelsum: 0.4058
+- Gen Len: 25.3851
+- Precision: 0.9131
+- Recall: 0.9117
+- F1: 0.9123
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 32
+- eval_batch_size: 16
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 128
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 8
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len | Precision | Recall | F1     |
+|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|:---------:|:------:|:------:|
+| 2.1501        | 1.0   | 781  | 1.7062          | 0.4566 | 0.209  | 0.3745 | 0.3744    | 25.4655 | 0.9082    | 0.9065 | 0.9072 |
+| 1.7722        | 2.0   | 1562 | 1.6314          | 0.4712 | 0.2226 | 0.3906 | 0.3904    | 25.4298 | 0.9107    | 0.909  | 0.9097 |
+| 1.7218        | 3.0   | 2343 | 1.5948          | 0.4776 | 0.2284 | 0.3965 | 0.3963    | 25.6569 | 0.9112    | 0.9103 | 0.9106 |
+| 1.6668        | 4.0   | 3125 | 1.5708          | 0.481  | 0.2316 | 0.4002 | 0.4       | 25.3451 | 0.9122    | 0.9107 | 0.9112 |
+| 1.6437        | 5.0   | 3906 | 1.5565          | 0.4844 | 0.2346 | 0.4034 | 0.4031    | 25.482  | 0.9127    | 0.9113 | 0.9118 |
+| 1.6186        | 6.0   | 4687 | 1.5476          | 0.4852 | 0.236  | 0.4047 | 0.4044    | 25.4191 | 0.9129    | 0.9115 | 0.912  |
+| 1.607         | 7.0   | 5468 | 1.5426          | 0.486  | 0.2367 | 0.4052 | 0.405     | 25.4949 | 0.9129    | 0.9118 | 0.9122 |
+| 1.5972        | 8.0   | 6248 | 1.5409          | 0.4869 | 0.2373 | 0.406  | 0.4058    | 25.3851 | 0.9131    | 0.9117 | 0.9123 |
+
+
+### Framework versions
+
+- Transformers 4.36.0
+- Pytorch 2.0.1+cu117
+- Datasets 2.14.5
+- Tokenizers 0.15.0

generation_config.json

@@ -0,0 +1,11 @@
+{
+  "bos_token_id": 0,
+  "decoder_start_token_id": 0,
+  "eos_token_id": 1,
+  "forced_eos_token_id": 1,
+  "length_penalty": 0.6,
+  "max_length": 64,
+  "num_beams": 8,
+  "pad_token_id": 0,
+  "transformers_version": "4.36.0"
+}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f9b403d20162a5df89e0f6f31342d27f05b1c635c27f53a921f590eb0410fa38
+oid sha256:d70b29dca24ea44a3385de0ff2a16ba9a09606655e4dc6b75d27a60f1d91286e
 size 2279458540

runs/Mar24_22-33-25_gde8xrctr1711261266898-zjnhj/events.out.tfevents.1711290808.gde8xrctr1711261266898-zjnhj.20316.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b8bdeb01063ce994c188f6767b1dde327a6efe287d409c5e77c26072b4cd0a5c
-size 11889
+oid sha256:b23cc8aaa0ff4034c4d415d87175541521978236ff90641e567034b84145750f
+size 12917