Model save

README.md

@@ -1,7 +1,7 @@
 ---
-base_model: mistralai/Mistral-Small-Instruct-2409
+base_model: RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter4
 library_name: transformers
-model_name: self-correct_mistral-small-it_mMQA_dpo_iter1_beta.05
+model_name: self-correct_mistral-small-it_mMQA_dpo_iter5
 tags:
 - generated_from_trainer
 - trl
@@ -9,9 +9,9 @@ tags:
 licence: license
 ---
 
-# Model Card for self-correct_mistral-small-it_mMQA_dpo_iter1_beta.05
+# Model Card for self-correct_mistral-small-it_mMQA_dpo_iter5
 
-This model is a fine-tuned version of [mistralai/Mistral-Small-Instruct-2409](https://huggingface.co/mistralai/Mistral-Small-Instruct-2409).
+This model is a fine-tuned version of [RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter4](https://huggingface.co/RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter4).
 It has been trained using [TRL](https://github.com/huggingface/trl).
 
 ## Quick start
@@ -20,14 +20,14 @@ It has been trained using [TRL](https://github.com/huggingface/trl).
 from transformers import pipeline
 
 question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter1_beta.05", device="cuda")
+generator = pipeline("text-generation", model="RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter5", device="cuda")
 output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
 print(output["generated_text"])
 ```
 
 ## Training procedure
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/yyr/huggingface/runs/o4gfqsnq)
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/yyr/huggingface/runs/astknhdc)
 
 This model was trained with DPO, a method introduced in [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://huggingface.co/papers/2305.18290).
 

last_checkpoint/config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "mistralai/Mistral-Small-Instruct-2409",
+  "_name_or_path": "RyanYr/self-correct_mistral-small-it_mMQA_dpo_iter4",
   "architectures": [
     "MistralForCausalLM"
   ],

last_checkpoint/model-00001-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b5d1196c2e8709e991f55d87f7a0a70c66eae35fcc18262addd62a99724752bc
+oid sha256:3c31c3a2768071f581cbccf37efd97d59877b893d382af83dd4afbcfb3082e75
 size 4882311064

last_checkpoint/model-00002-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:81bb6435abda482131efd56bd9f8ffda07827fbeee8c91ddcea289871dc41ba7
+oid sha256:3ddf52c556e06c7abcda48ec13ba7d3631b96d2761dd4caa738c8434f011e1d3
 size 4983012160

last_checkpoint/model-00003-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6c282b24f960342a53ae7e78a729978320e331667321bbc265e2b83794daca7c
+oid sha256:067503063301fca44db66c50eaba52dbaddd2783253cc90b0ae545bcbc82e70f
 size 4957821336

last_checkpoint/model-00004-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d59cc369a137f03d39d67d90dcc81174a31941ea44b56852a6554b6f26438cf5
+oid sha256:63c4f9420a7f2db173904609a44f76e23a7c35c8acb3cf04b7920d5efddee95a
 size 4882323744

last_checkpoint/model-00005-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dd73870967da1e197a682b08de7e637391efce1a1e9dda0ac4a97bbbe3fb5adb
+oid sha256:df900aa99d6bf6d9caf2586eae913724a07fb1a7b85053b5623999ec5bed993d
 size 4983012192

last_checkpoint/model-00006-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:12b4fed1ad562efb81c54a0deb5e979eeb2770fdcfef649eff6b825c259d0e6c
+oid sha256:c854eaf16dde965b51aec8159b371c3053118676119949c4011b2bc57909284c
 size 4957821336

last_checkpoint/model-00007-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c0bc65b2f0966d0eaaf351c3d16ef6fb9249479b0c96272c35279d0a58a492ec
+oid sha256:5158cfe71311e7cd5ad2cafb83a72a028c64968b44bab80c9e6d892777c2a74c
 size 4882323744

last_checkpoint/model-00008-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:70e7bf89d90290fde28b5e13aae84b3becd5aeddfe25bb85b4b9364b515ce45a
+oid sha256:07a2b05940d3a5fceb43ceaae485cb1254fcc7d00fb4e362d03eb334dc6fba77
 size 4983012192

last_checkpoint/model-00009-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:927ac7c0ea72e96bbd68edf8f6040c981fc5fb2cce32844d75c0c6bab4e04333
+oid sha256:973b313ed3c3401afaddb2c9dd87ac4634227ea3b6d3858bd026ceb5a13041b2
 size 4983011344

last_checkpoint/training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3563ab0befb4d64cea2ad9869f75176474c735c329f7161ddcf2683a3861e78c
-size 8056
+oid sha256:81c3725711de0f6dd0d0e3c8789e334fa06110e34ea549423cccf59da4534392
+size 7992