Commit
•
c7248ad
1
Parent(s):
c919494
Update README.md
Browse files
README.md
CHANGED
|
@@ -34,8 +34,6 @@ dataset using QloRA. The model has been trained for 1 epoch on 1x A40 GPU. The e
|
|
| 34 |
|
| 35 |
This experiment was performed using **[Transformer-heads library](https://github.com/center-for-humans-and-machines/transformer-heads/tree/main)**
|
| 36 |
|
| 37 |
-
</details><br>
|
| 38 |
-
|
| 39 |
### Training Script
|
| 40 |
|
| 41 |
The training script for attaching a new transformer head for classification task using QLoRA is following:
|
|
@@ -53,6 +51,12 @@ For evaluating the transformer head that has been attached to the base model, yo
|
|
| 53 |
|
| 54 |
The following hyperparameters were used during training:
|
| 55 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
* output_dir="emotion_linear_probe",
|
| 57 |
* learning_rate=0.00002,
|
| 58 |
* num_train_epochs=train_epochs,
|
|
|
|
| 34 |
|
| 35 |
This experiment was performed using **[Transformer-heads library](https://github.com/center-for-humans-and-machines/transformer-heads/tree/main)**
|
| 36 |
|
|
|
|
|
|
|
| 37 |
### Training Script
|
| 38 |
|
| 39 |
The training script for attaching a new transformer head for classification task using QLoRA is following:
|
|
|
|
| 51 |
|
| 52 |
The following hyperparameters were used during training:
|
| 53 |
|
| 54 |
+
train_epochs = 1
|
| 55 |
+
eval_epochs = 1
|
| 56 |
+
logging_steps = 1
|
| 57 |
+
train_batch_size = 4
|
| 58 |
+
eval_batch_size = 4
|
| 59 |
+
|
| 60 |
* output_dir="emotion_linear_probe",
|
| 61 |
* learning_rate=0.00002,
|
| 62 |
* num_train_epochs=train_epochs,
|