| license: mit | |
| datasets: | |
| - HuggingFaceH4/ultrafeedback_binarized | |
| base_model: | |
| - princeton-nlp/Llama-3-Base-8B-SFT | |
| This is an aligned model based on princeton-nlp/Llama-3-Base-8B-SFT. The alignment loss has two parts, preference loss and a regularization term. The alignment loss does not require the assumption of an existing underlying reward model. Moreover, the distribution's ``softness" is adjustable via the softmax exponent, an algorithm parameter. | |
| More information will be added later. |