Model save

README.md

@@ -0,0 +1,93 @@
+---
+tags:
+- generated_from_trainer
+datasets:
+- data
+metrics:
+- bleu
+model-index:
+- name: mbart-en-id-smaller-indo-amr-generation-fted-with-prefix
+  results:
+  - task:
+      name: Sequence-to-sequence Language Modeling
+      type: text2text-generation
+    dataset:
+      name: data
+      type: data
+      config: default
+      split: validation
+      args: default
+    metrics:
+    - name: Bleu
+      type: bleu
+      value: 13.717
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# mbart-en-id-smaller-indo-amr-generation-fted-with-prefix
+
+This model was trained from scratch on the data dataset.
+It achieves the following results on the evaluation set:
+- Loss: 2.3974
+- Bleu: 13.717
+- Gen Len: 36.5221
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-07
+- train_batch_size: 2
+- eval_batch_size: 2
+- seed: 42
+- gradient_accumulation_steps: 12
+- total_train_batch_size: 24
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: polynomial
+- lr_scheduler_warmup_steps: 200
+- num_epochs: 16.0
+- label_smoothing_factor: 0.1
+
+### Training results
+
+| Training Loss | Epoch   | Step  | Bleu    | Gen Len  | Validation Loss |
+|:-------------:|:-------:|:-----:|:-------:|:--------:|:---------------:|
+| 3.0219        | 0.9999  | 3869  | 0.0741  | 114.8177 | 2.9798          |
+| 2.8978        | 2.0     | 7739  | 0.0747  | 113.0081 | 2.8610          |
+| 2.8109        | 2.9999  | 11608 | 0.0795  | 111.475  | 2.7648          |
+| 2.7623        | 4.0     | 15478 | 0.1685  | 105.7747 | 2.6956          |
+| 2.7116        | 4.9999  | 19347 | 0.5081  | 92.4187  | 2.6404          |
+| 2.6331        | 5.9999  | 23214 | 1.6991  | 66.9245  | 2.5961          |
+| 2.5716        | 7.0     | 27084 | 5.2201  | 46.1405  | 2.5611          |
+| 2.5943        | 7.9999  | 30953 | 8.0263  | 40.7538  | 2.5300          |
+| 2.5622        | 9.0     | 34823 | 10.2353 | 38.2607  | 2.5050          |
+| 2.537         | 9.9999  | 38692 | 11.3364 | 36.0732  | 2.4840          |
+| 2.5345        | 11.0    | 42562 | 12.1716 | 36.4367  | 2.4645          |
+| 2.4706        | 11.9999 | 46428 | 2.4479  | 12.51    | 37.4146         |
+| 2.4558        | 13.0    | 50298 | 2.4330  | 12.8144  | 37.2979         |
+| 2.4125        | 13.9999 | 54167 | 2.4199  | 13.0772  | 37.0436         |
+| 2.4053        | 15.0    | 58037 | 2.4081  | 13.5764  | 36.1492         |
+| 2.439         | 15.9994 | 61904 | 2.3974  | 13.717   | 36.5221         |
+
+
+### Framework versions
+
+- Transformers 4.44.0
+- Pytorch 2.4.0+cu121
+- Datasets 2.20.0
+- Tokenizers 0.19.1

generation_config.json

@@ -0,0 +1,12 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "decoder_start_token_id": 2,
+  "early_stopping": true,
+  "eos_token_id": 2,
+  "forced_eos_token_id": 2,
+  "max_length": 200,
+  "num_beams": 5,
+  "pad_token_id": 1,
+  "transformers_version": "4.44.0"
+}

last-checkpoint/model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f5610f104bbe5c71c2c8986a36c189e0fd7e9345d34c3ad78d767d6afdfb463c
+oid sha256:b35a3bc2ac180ed070b42029c9a9dd327a1a9559e81df276f329c07eb21d04fc
 size 1575259780

last-checkpoint/optimizer.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7d7ef396422aa6c019a5e69eda3ff44645515da108954efa0d94036fd3d1bc22
+oid sha256:ba79276529acf6b7fcac21ab5be5fc5756c900e697415161ce71f7759f9fa8e0
 size 3150397656

last-checkpoint/rng_state.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:29aef665b26cb1458ee97ee69d3ddd4704a10f9d74c0acba81e307397efc04fa
+oid sha256:2e52ca2f5b1048c2984d9cb01ff8bc5c06ec7e6e1ac850eb54ef8fe7147dcf65
 size 14244

last-checkpoint/scheduler.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a8adff87233e7c0e142f515f7a41b0a9ab9d0ffb3771224e0f1d596be2a78b03
+oid sha256:df05e968bcb6cb9f8c607bdaf90fbac1131121f0efd29f2e5e7bc42c79c2d577
 size 1064

run-2024-10-28T15:22:08+00:00.log

@@ -5374,3 +5374,7 @@ Non-default generation parameters: {'max_length': 200, 'early_stopping': True, '
 [WARNING|trainer.py:2764] 2024-10-29 00:01:14,086 >> There were missing keys in the checkpoint model loaded: ['model.encoder.embed_tokens.weight', 'model.decoder.embed_tokens.weight', 'lm_head.weight'].
 
                                                       
 
+[WARNING|configuration_utils.py:448] 2024-10-29 00:01:47,338 >> Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
+Non-default generation parameters: {'max_length': 200, 'early_stopping': True, 'num_beams': 5, 'forced_eos_token_id': 2}
+[WARNING|configuration_utils.py:448] 2024-10-29 00:01:58,952 >> Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
+Non-default generation parameters: {'max_length': 200, 'early_stopping': True, 'num_beams': 5, 'forced_eos_token_id': 2}