complete: train_size: {train_size}, batch_size: {batch_size}, per_epoch_steps: {per_epoch_steps}, epochs: {epochs}, epoch_total_steps: {epoch_total_steps}

README.md

@@ -0,0 +1,77 @@
+---
+license: mit
+base_model: openai-community/gpt2-large
+tags:
+- generated_from_trainer
+metrics:
+- rouge
+model-index:
+- name: gpt2-large-coedit
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# gpt2-large-coedit
+
+This model is a fine-tuned version of [openai-community/gpt2-large](https://huggingface.co/openai-community/gpt2-large) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.9215
+- Rouge1: 0.4818
+- Rouge2: 0.3649
+- Rougel: 0.4555
+- Rougelsum: 0.4643
+- Sacreblue: 19.1714
+- Memory Used: 68475.5
+- Cuda Allocated: 3082.6328
+- Cuda Reserved: 61060.0
+- Ram Usage: 13976.5117
+- Em: 0.0
+- Gen Len: 82.1798
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 150
+- eval_batch_size: 4
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 600
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 1
+- num_epochs: 2
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Sacreblue | Memory Used | Cuda Allocated | Cuda Reserved | Ram Usage  | Em  | Gen Len |
+|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:---------:|:-----------:|:--------------:|:-------------:|:----------:|:---:|:-------:|
+| 0.8724        | 0.47  | 50   | 1.0274          | 0.4653 | 0.3509 | 0.4382 | 0.4459    | 19.0412   | 68475.5     | 3082.605       | 61060.0       | 5708.957   | 0.0 | 82.0895 |
+| 0.7407        | 0.94  | 100  | 0.9499          | 0.4825 | 0.3651 | 0.4557 | 0.4656    | 19.2975   | 68475.5     | 3082.6152      | 61060.0       | 13842.9336 | 0.0 | 81.3952 |
+| 0.6964        | 1.41  | 150  | 0.9318          | 0.4783 | 0.3627 | 0.452  | 0.4605    | 19.418    | 68475.5     | 3082.6182      | 61060.0       | 13958.2773 | 0.0 | 81.0295 |
+| 0.6846        | 1.88  | 200  | 0.9215          | 0.4818 | 0.3649 | 0.4555 | 0.4643    | 19.1714   | 68475.5     | 3082.6328      | 61060.0       | 13976.5117 | 0.0 | 82.1798 |
+
+
+### Framework versions
+
+- Transformers 4.39.3
+- Pytorch 2.2.2
+- Datasets 2.18.0
+- Tokenizers 0.15.2

config.json

@@ -0,0 +1,39 @@
+{
+  "_name_or_path": "openai-community/gpt2-large",
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 1280,
+  "n_head": 20,
+  "n_inner": null,
+  "n_layer": 36,
+  "n_positions": 1024,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.39.3",
+  "use_cache": false,
+  "vocab_size": 50257
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "max_new_tokens": 350,
+  "pad_token_id": 50256,
+  "padding_side": "left",
+  "transformers_version": "4.39.3"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9b42507ab4940ddee24f64c2aacc25de3566da119c0b67c0b8662f7dc477b6e
+size 3096165928

runs/Apr21_00-17-16_workstation/events.out.tfevents.1713658636.workstation.89102.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88e4b19ad59505aa8cce799015df9e8d916e4a5847c959a59ccd02b253983e34
+size 10379

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "pad_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,20 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2edbeab06b2e8fef6a91e7becae217b90f06c61c251ad80cb955b2fad9ed5d91
+size 5048