raidhon commited on
Commit
df66f02
1 Parent(s): d4d334b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -143,7 +143,7 @@ model-index:
143
  # 🧙 Coven 7B 128K ORPO
144
 
145
 
146
- Coven 7B 128K is an improved iteration of Mistral-7B-Instruct-v0.2, refined to expand processing capabilities and refine language model preferences. This model includes a significantly increased context constraint of 128K tokens using the Yarn technique, which allows for more extensive data processing and understanding of complex language scenarios. In addition, the Coven 7B 128K ORPO 32K tokenization uses the innovative ORPO (Monolithic Preference Optimization without Reference Model) technology. ORPO simplifies the fine-tuning process by directly optimizing the odds ratio to distinguish between favorable and unfavorable generation styles, effectively improving model performance without the need for an additional preference alignment step.
147
 
148
 
149
  ### Eval
 
143
  # 🧙 Coven 7B 128K ORPO
144
 
145
 
146
+ Coven 7B 128K is an improved iteration of Mistral-7B-Instruct-v0.2, refined to expand processing capabilities and refine language model preferences. This model includes a significantly increased context constraint of 128K tokens using the [Yarn](https://github.com/jquesnelle/yarn) technique, which allows for more extensive data processing and understanding of complex language scenarios. In addition, the Coven 7B ORPO 128K tokenization uses the innovative ORPO (Monolithic Preference Optimization without Reference Model) technology. ORPO simplifies the fine-tuning process by directly optimizing the odds ratio to distinguish between favorable and unfavorable generation styles, effectively improving model performance without the need for an additional preference alignment step.
147
 
148
 
149
  ### Eval