End of training

README.md

@@ -0,0 +1,70 @@
+---
+library_name: transformers
+tags:
+- generated_from_trainer
+model-index:
+- name: Beit_PhoBart
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# Beit_PhoBart
+
+This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.5865
+- Bleu-4: 0.0954
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 32
+- eval_batch_size: 32
+- seed: 42
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 64
+- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 300
+- num_epochs: 100
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Bleu-4 |
+|:-------------:|:-----:|:----:|:---------------:|:------:|
+| No log        | 1.0   | 236  | 1.7574          | 0.0672 |
+| 1.9578        | 2.0   | 472  | 1.6226          | 0.0748 |
+| 1.5495        | 3.0   | 708  | 1.5523          | 0.0796 |
+| 1.3763        | 4.0   | 944  | 1.4998          | 0.0821 |
+| 1.3763        | 5.0   | 1180 | 1.4815          | 0.0881 |
+| 1.2355        | 6.0   | 1416 | 1.4835          | 0.0875 |
+| 1.1027        | 7.0   | 1652 | 1.4790          | 0.0926 |
+| 0.9865        | 8.0   | 1888 | 1.5128          | 0.0925 |
+| 0.884         | 9.0   | 2124 | 1.5453          | 0.0956 |
+| 0.884         | 10.0  | 2360 | 1.5865          | 0.0954 |
+
+
+### Framework versions
+
+- Transformers 4.46.2
+- Pytorch 2.5.1+cu121
+- Datasets 3.1.0
+- Tokenizers 0.20.3

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 50256,
+  "decoder_start_token_id": 50257,
+  "early_stopping": true,
+  "eos_token_id": 50257,
+  "length_penalty": 2.0,
+  "max_length": 29,
+  "no_repeat_ngram_size": 3,
+  "num_beams": 4,
+  "pad_token_id": 50258,
+  "transformers_version": "4.46.2",
+  "use_cache": false
+}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4eae45e1766c8299fdbf0ad10440e471c3cdf55d995d3ce9b1edc697db561c05
+oid sha256:ac90bf03b5adc2175e0892d406a6e233ddb858bdf40d8cd52f1fd0ee189a473a
 size 1400816568