Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
| 1 |
This model is a result of merging three Orca2-13B models with itself using 'mergekit-legacy'. Merge parameters were passthrough for one and --weight 0.5 --density 0.5 for the second merge.
|
| 2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
license: ms-pl
|
| 5 |
---
|
|
|
|
| 1 |
This model is a result of merging three Orca2-13B models with itself using 'mergekit-legacy'. Merge parameters were passthrough for one and --weight 0.5 --density 0.5 for the second merge.
|
| 2 |
|
| 3 |
+
|
| 4 |
+
This merged model showed marginal improvement in perplexity scores:
|
| 5 |
+
|
| 6 |
+
The perplexity for Orca-2-13b is: 7.595028877258301
|
| 7 |
+
The perplexity for orca2-26B-self-merge is: 7.550178050994873
|
| 8 |
+
The perplexity for orca2-39B-self-merge is:
|
| 9 |
+
|
| 10 |
+
|
| 11 |
---
|
| 12 |
license: ms-pl
|
| 13 |
---
|