--- license: mit library_name: peft tags: - trl - reward-trainer - generated_from_trainer metrics: - accuracy base_model: openai-community/gpt2-large model-index: - name: RM-HH-GPT2Large_helpful_gpt3_loraR64_40000_gpt2-large_shuffleFalse_extractchosenFalse results: [] --- # RM-HH-GPT2Large_helpful_gpt3_loraR64_40000_gpt2-large_shuffleFalse_extractchosenFalse This model is a fine-tuned version of [openai-community/gpt2-large](https://huggingface.co/openai-community/gpt2-large) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.0335 - Accuracy: 0.9906 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1.41e-05 - train_batch_size: 1 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 4 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 2.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:-----:|:-----:|:---------------:|:--------:| | 0.6956 | 0.02 | 250 | 0.5812 | 0.7384 | | 0.6388 | 0.04 | 500 | 0.3891 | 0.9145 | | 0.5931 | 0.06 | 750 | 0.2579 | 0.9580 | | 0.5705 | 0.08 | 1000 | 0.1982 | 0.9750 | | 0.5449 | 0.1 | 1250 | 0.1534 | 0.9797 | | 0.577 | 0.13 | 1500 | 0.1506 | 0.9821 | | 0.5225 | 0.15 | 1750 | 0.1299 | 0.9836 | | 0.5516 | 0.17 | 2000 | 0.1285 | 0.9866 | | 0.5528 | 0.19 | 2250 | 0.1244 | 0.9870 | | 0.579 | 0.21 | 2500 | 0.1299 | 0.9881 | | 0.5386 | 0.23 | 2750 | 0.1140 | 0.9881 | | 0.5427 | 0.25 | 3000 | 0.1057 | 0.9885 | | 0.5502 | 0.27 | 3250 | 0.1000 | 0.9889 | | 0.5309 | 0.29 | 3500 | 0.0818 | 0.9895 | | 0.558 | 0.31 | 3750 | 0.0966 | 0.9896 | | 0.5523 | 0.33 | 4000 | 0.0833 | 0.9898 | | 0.545 | 0.36 | 4250 | 0.0920 | 0.9902 | | 0.5402 | 0.38 | 4500 | 0.0928 | 0.9898 | | 0.5271 | 0.4 | 4750 | 0.0824 | 0.9902 | | 0.5613 | 0.42 | 5000 | 0.0903 | 0.9915 | | 0.5064 | 0.44 | 5250 | 0.0723 | 0.9913 | | 0.5714 | 0.46 | 5500 | 0.0738 | 0.9915 | | 0.5285 | 0.48 | 5750 | 0.0756 | 0.9908 | | 0.5311 | 0.5 | 6000 | 0.0757 | 0.9906 | | 0.5205 | 0.52 | 6250 | 0.0730 | 0.9895 | | 0.5311 | 0.54 | 6500 | 0.0729 | 0.9904 | | 0.5209 | 0.57 | 6750 | 0.0666 | 0.9896 | | 0.5529 | 0.59 | 7000 | 0.0795 | 0.9904 | | 0.5495 | 0.61 | 7250 | 0.0698 | 0.9910 | | 0.5184 | 0.63 | 7500 | 0.0695 | 0.9902 | | 0.5609 | 0.65 | 7750 | 0.0722 | 0.9904 | | 0.5024 | 0.67 | 8000 | 0.0656 | 0.9904 | | 0.5536 | 0.69 | 8250 | 0.0779 | 0.9889 | | 0.5402 | 0.71 | 8500 | 0.0715 | 0.9893 | | 0.5204 | 0.73 | 8750 | 0.0681 | 0.9896 | | 0.544 | 0.75 | 9000 | 0.0700 | 0.9896 | | 0.5502 | 0.77 | 9250 | 0.0722 | 0.9902 | | 0.5334 | 0.8 | 9500 | 0.0650 | 0.9910 | | 0.5229 | 0.82 | 9750 | 0.0606 | 0.9900 | | 0.5235 | 0.84 | 10000 | 0.0525 | 0.9906 | | 0.534 | 0.86 | 10250 | 0.0623 | 0.9895 | | 0.5314 | 0.88 | 10500 | 0.0561 | 0.9904 | | 0.5311 | 0.9 | 10750 | 0.0503 | 0.9902 | | 0.5457 | 0.92 | 11000 | 0.0515 | 0.9910 | | 0.548 | 0.94 | 11250 | 0.0589 | 0.9910 | | 0.5504 | 0.96 | 11500 | 0.0612 | 0.9908 | | 0.5102 | 0.98 | 11750 | 0.0501 | 0.9908 | | 0.5197 | 1.0 | 12000 | 0.0505 | 0.9913 | | 0.5406 | 1.03 | 12250 | 0.0458 | 0.9908 | | 0.5372 | 1.05 | 12500 | 0.0468 | 0.9908 | | 0.4972 | 1.07 | 12750 | 0.0429 | 0.9910 | | 0.5059 | 1.09 | 13000 | 0.0422 | 0.9906 | | 0.536 | 1.11 | 13250 | 0.0462 | 0.9900 | | 0.5116 | 1.13 | 13500 | 0.0408 | 0.9904 | | 0.5504 | 1.15 | 13750 | 0.0479 | 0.9908 | | 0.5393 | 1.17 | 14000 | 0.0462 | 0.9908 | | 0.511 | 1.19 | 14250 | 0.0426 | 0.9908 | | 0.5059 | 1.21 | 14500 | 0.0403 | 0.9906 | | 0.5324 | 1.23 | 14750 | 0.0381 | 0.9906 | | 0.5227 | 1.26 | 15000 | 0.0368 | 0.9906 | | 0.5377 | 1.28 | 15250 | 0.0442 | 0.9904 | | 0.5269 | 1.3 | 15500 | 0.0446 | 0.9906 | | 0.5088 | 1.32 | 15750 | 0.0487 | 0.9904 | | 0.5271 | 1.34 | 16000 | 0.0474 | 0.9908 | | 0.4952 | 1.36 | 16250 | 0.0377 | 0.9915 | | 0.5201 | 1.38 | 16500 | 0.0392 | 0.9906 | | 0.5316 | 1.4 | 16750 | 0.0431 | 0.9908 | | 0.5186 | 1.42 | 17000 | 0.0421 | 0.9900 | | 0.4963 | 1.44 | 17250 | 0.0366 | 0.9908 | | 0.5324 | 1.46 | 17500 | 0.0392 | 0.9906 | | 0.5257 | 1.49 | 17750 | 0.0392 | 0.9911 | | 0.4908 | 1.51 | 18000 | 0.0348 | 0.9910 | | 0.5186 | 1.53 | 18250 | 0.0371 | 0.9906 | | 0.5385 | 1.55 | 18500 | 0.0385 | 0.9906 | | 0.5267 | 1.57 | 18750 | 0.0370 | 0.9910 | | 0.5294 | 1.59 | 19000 | 0.0372 | 0.9906 | | 0.5243 | 1.61 | 19250 | 0.0360 | 0.9908 | | 0.5414 | 1.63 | 19500 | 0.0376 | 0.9906 | | 0.5171 | 1.65 | 19750 | 0.0403 | 0.9904 | | 0.5081 | 1.67 | 20000 | 0.0363 | 0.9908 | | 0.543 | 1.7 | 20250 | 0.0353 | 0.9908 | | 0.5121 | 1.72 | 20500 | 0.0341 | 0.9910 | | 0.5047 | 1.74 | 20750 | 0.0330 | 0.9908 | | 0.5386 | 1.76 | 21000 | 0.0327 | 0.9911 | | 0.5261 | 1.78 | 21250 | 0.0341 | 0.9910 | | 0.4973 | 1.8 | 21500 | 0.0329 | 0.9913 | | 0.5185 | 1.82 | 21750 | 0.0329 | 0.9911 | | 0.5215 | 1.84 | 22000 | 0.0325 | 0.9911 | | 0.4922 | 1.86 | 22250 | 0.0314 | 0.9911 | | 0.5354 | 1.88 | 22500 | 0.0327 | 0.9908 | | 0.5489 | 1.9 | 22750 | 0.0337 | 0.9911 | | 0.538 | 1.93 | 23000 | 0.0336 | 0.9913 | | 0.508 | 1.95 | 23250 | 0.0335 | 0.9910 | | 0.5316 | 1.97 | 23500 | 0.0333 | 0.9910 | | 0.5496 | 1.99 | 23750 | 0.0335 | 0.9906 | ### Framework versions - PEFT 0.9.0 - Transformers 4.38.2 - Pytorch 2.1.2 - Datasets 2.18.0 - Tokenizers 0.15.2