End of training

README.md

@@ -1,6 +1,6 @@
 ---
 license: apache-2.0
-base_model: distilbert-base-cased
+base_model: distilbert-base-uncased
 tags:
 - generated_from_trainer
 model-index:
@@ -13,9 +13,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # da_distilbert
 
-This model is a fine-tuned version of [distilbert-base-cased](https://huggingface.co/distilbert-base-cased) on the None dataset.
+This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.6089
+- Loss: 1.7151
 
 ## Model description
 
@@ -40,20 +40,22 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 8
+- num_epochs: 10
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| No log        | 1.0   | 130  | 1.9587          |
-| No log        | 2.0   | 260  | 1.8275          |
-| No log        | 3.0   | 390  | 1.7576          |
-| 1.9627        | 4.0   | 520  | 1.7045          |
-| 1.9627        | 5.0   | 650  | 1.6049          |
-| 1.9627        | 6.0   | 780  | 1.6452          |
-| 1.9627        | 7.0   | 910  | 1.5920          |
-| 1.6873        | 8.0   | 1040 | 1.6354          |
+| No log        | 1.0   | 130  | 2.1034          |
+| No log        | 2.0   | 260  | 1.9547          |
+| No log        | 3.0   | 390  | 1.8459          |
+| 2.1418        | 4.0   | 520  | 1.8499          |
+| 2.1418        | 5.0   | 650  | 1.7874          |
+| 2.1418        | 6.0   | 780  | 1.7771          |
+| 2.1418        | 7.0   | 910  | 1.7605          |
+| 1.8295        | 8.0   | 1040 | 1.7202          |
+| 1.8295        | 9.0   | 1170 | 1.6926          |
+| 1.8295        | 10.0  | 1300 | 1.7350          |
 
 
 ### Framework versions

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "distilbert-base-cased",
+  "_name_or_path": "distilbert-base-uncased",
   "activation": "gelu",
   "architectures": [
     "DistilBertForMaskedLM"
@@ -13,7 +13,6 @@
   "model_type": "distilbert",
   "n_heads": 12,
   "n_layers": 6,
-  "output_past": true,
   "pad_token_id": 0,
   "qa_dropout": 0.1,
   "seq_classif_dropout": 0.2,
@@ -21,5 +20,5 @@
   "tie_weights_": true,
   "torch_dtype": "float32",
   "transformers_version": "4.40.1",
-  "vocab_size": 28996
+  "vocab_size": 30522
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fb8f9b788a7dd1858ff3099aceadd76027c6663bc8abf9382b1bed09ac274180
-size 263260784
+oid sha256:eff1a41cfde9865d4c2f05171292cd2bcdecb2c0f7d4ed7a7b71b44f6447802c
+size 267954768

tokenizer_config.json

@@ -43,9 +43,9 @@
   },
   "clean_up_tokenization_spaces": true,
   "cls_token": "[CLS]",
-  "do_lower_case": false,
+  "do_lower_case": true,
   "mask_token": "[MASK]",
-  "model_max_length": 1000000000000000019884624838656,
+  "model_max_length": 512,
   "pad_token": "[PAD]",
   "sep_token": "[SEP]",
   "strip_accents": null,

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:51a709fd88a40cd7d65704303344c951e970172f3a14ecb222a9b9c05ed6ebba
-size 4984
+oid sha256:b52f3b68a04ebd189bd9cf0bfeec1c67d27cf6c66420b49ba2bc9ab91a56e5b0
+size 4920