Model save

README.md

@@ -1,7 +1,7 @@
 ---
-base_model: RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter3
+base_model: RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter1
 library_name: transformers
-model_name: self-correct_mistral-small-it_mMQA_dpo_iter4_beta.05
+model_name: self-correct_mistral-small-it_mMQA_dpo_iter2_beta.05
 tags:
 - generated_from_trainer
 - trl
@@ -9,9 +9,9 @@ tags:
 licence: license
 ---
 
-# Model Card for self-correct_mistral-small-it_mMQA_dpo_iter4_beta.05
+# Model Card for self-correct_mistral-small-it_mMQA_dpo_iter2_beta.05
 
-This model is a fine-tuned version of [RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter3](https://huggingface.co/RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter3).
+This model is a fine-tuned version of [RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter1](https://huggingface.co/RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter1).
 It has been trained using [TRL](https://github.com/huggingface/trl).
 
 ## Quick start
@@ -20,14 +20,14 @@ It has been trained using [TRL](https://github.com/huggingface/trl).
 from transformers import pipeline
 
 question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter4_beta.05", device="cuda")
+generator = pipeline("text-generation", model="RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter2_beta.05", device="cuda")
 output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
 print(output["generated_text"])
 ```
 
 ## Training procedure
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/yyr/huggingface/runs/9v4u4jit)
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/yyr/huggingface/runs/9ec28lm9)
 
 This model was trained with DPO, a method introduced in [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://huggingface.co/papers/2305.18290).
 

last_checkpoint/config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter3",
+  "_name_or_path": "RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter1",
   "architectures": [
     "MistralForCausalLM"
   ],

last_checkpoint/model-00001-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1ce56c04a7874e4526e2fe334e4daf76de4ad4906910397d6fda4723edc26bfe
+oid sha256:464ac8437e48d586d41a333ed6a7e4b2b39b5dcd8bbd73abba2faefd3517cf26
 size 4882311064

last_checkpoint/model-00002-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9ad99d58e44f3950c541a7a3e2124abe73ba2d8826d1a36bbe9f10bfec485c5b
+oid sha256:080fb99eefb3790b94cfec04ca25e9e75c7f003424b2d01e674d1f9cf616aef9
 size 4983012160

last_checkpoint/model-00003-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:543c30a9770268d5a83da5933f180d5615de61a6bd5d0ef06890c6147d8a8ef1
+oid sha256:c2aa87c471c78ed5711ccaf3c4e7f3a83b3fc2e1c92c66a22ea2d5829c8ac45c
 size 4957821336

last_checkpoint/model-00004-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7e9840f56194de65753cc5d1e17180dc43372a31ea362cc4057c34e760e6bb7c
+oid sha256:a1232d5d34f02adf3065664ad362051e6165a73a96b374a2d390a328a620d5ff
 size 4882323744

last_checkpoint/model-00005-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:eaa6b08c20bbe768ad0f203937bab37fe7086311c0393f8b0d099c0c0f887a2f
+oid sha256:8c6978e76c9f29ab95f800bb3eff65b8e9a19381ab3cc9ed6f11e19dff6c4796
 size 4983012192

last_checkpoint/model-00006-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3c0e9207ab4ec54ff3ebf34b298c28813088ce536132cc7be47ff3c6a31fd0b1
+oid sha256:8463efa2f7cf5a8d03e3604e9eb1e2008acdd8c9ae67abc9f33dab27b97aceb0
 size 4957821336

last_checkpoint/model-00007-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f40eb60bb55571888feca2ddd30ed90b3a27ed4a248de8c2be7017bd07bdd756
+oid sha256:fa981da7eca667a0d10e694a87da0b71f02182796b003c52441600b23523ebce
 size 4882323744

last_checkpoint/model-00008-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:226801c226c219a40e695b7e5e84dc1e9f08cfb4e2606985242701b407201bb7
+oid sha256:6abe9dae918d6a2eeda8a07dc6fa2ca525391d5128b5c0bcdde2e21d12760905
 size 4983012192

last_checkpoint/model-00009-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b700e6a9059e61d84fcf2dd67269a79cc94869a9d1eb9ee5a656ebe3a228b878
+oid sha256:43589738ab4d826e22d2de7d8e7a803e7e6fe6bda0dfb64ff3b3c89c205ef863
 size 4983011344

last_checkpoint/training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:54d262dbc7511ad28b68a59cbd7ca46588a94f311178792d0cee28854ba7d066
+oid sha256:182dbfc5cfadd9e156149dd754da5a26bb1016076a06193570698711c2a26a26
 size 8056