Update README.md
Browse files
README.md
CHANGED
@@ -22,18 +22,23 @@ Introducing THE MODEL: **XABERIUS 34B v1-BETA** an *experimental* 34B LLaMa-Yi-3
|
|
22 |
Timeline:
|
23 |
* 05-Dec-2023 **v1-beta released**
|
24 |
* 08-Dec-2023 **Evaluation been "RUNNING" for 2 days.. no results yet**
|
|
|
|
|
|
|
25 |
|
26 |
| Model | Average | ARC (25-s) | HellaSwag (10-s) | MMLU (5-s) | TruthfulQA (MC) (0-s) | Winogrande (5-s) | GSM8K (5-s) |
|
27 |
| --- | --- | --- | --- | --- | --- | --- | --- |
|
28 |
| [fblgit/una-cybertron-7b-v1-fp16](https://huggingface.co/fblgit/una-cybertron-7b-v1-fp16) | **69.49** | **68.43** | **85.85** | 63.34 | **63.28** | **80.90** | **55.12** |
|
29 |
| [fblgit/una-cybertron-7b-v2-bf16](https://huggingface.co/fblgit/una-cybertron-7b-v2-bf16) | **69.67** | **68.26** | **85.?4** | 63.23 | **64.63** | **81.37** | **55.04** |
|
30 |
-
| [fblgit/una-xaberius-34b-v1beta](https://huggingface.co/fblgit/una-xaberius-34b-v1beta) | **74.
|
31 |
|
32 |
## Evaluations
|
33 |
|
34 |
- Scores **74.21** Outperforming former leader tigerbot-70b-chat and landing on #1 position of HuggingFace LeaderBoard: 08 December 2023.
|
35 |
- Scores **79.13** in MMLU, setting a new record not just for 34B but also for all OpenSource LLM's :)
|
36 |
|
|
|
|
|
37 |
## Model Details
|
38 |
|
39 |
Adiestrated with UNA: Uniform Neural Alignment technique (paper going out soon).
|
|
|
22 |
Timeline:
|
23 |
* 05-Dec-2023 **v1-beta released**
|
24 |
* 08-Dec-2023 **Evaluation been "RUNNING" for 2 days.. no results yet**
|
25 |
+
* 09-Dec-2023 **Evaluation been "FINISHED", confirming #1 spot** outperforming the contaminated-disqualified tigerbot :)
|
26 |
+
|
27 |
+
Sidenote: Tests took 19H to run, wonder what happened in the 48H that HF held this one.. interim releasing manually other results??..
|
28 |
|
29 |
| Model | Average | ARC (25-s) | HellaSwag (10-s) | MMLU (5-s) | TruthfulQA (MC) (0-s) | Winogrande (5-s) | GSM8K (5-s) |
|
30 |
| --- | --- | --- | --- | --- | --- | --- | --- |
|
31 |
| [fblgit/una-cybertron-7b-v1-fp16](https://huggingface.co/fblgit/una-cybertron-7b-v1-fp16) | **69.49** | **68.43** | **85.85** | 63.34 | **63.28** | **80.90** | **55.12** |
|
32 |
| [fblgit/una-cybertron-7b-v2-bf16](https://huggingface.co/fblgit/una-cybertron-7b-v2-bf16) | **69.67** | **68.26** | **85.?4** | 63.23 | **64.63** | **81.37** | **55.04** |
|
33 |
+
| [fblgit/una-xaberius-34b-v1beta](https://huggingface.co/fblgit/una-xaberius-34b-v1beta) | **74.18** | **70.39** | **86.77** | **78.15** | **61.45** | **84.93** | **63.38** |
|
34 |
|
35 |
## Evaluations
|
36 |
|
37 |
- Scores **74.21** Outperforming former leader tigerbot-70b-chat and landing on #1 position of HuggingFace LeaderBoard: 08 December 2023.
|
38 |
- Scores **79.13** in MMLU, setting a new record not just for 34B but also for all OpenSource LLM's :)
|
39 |
|
40 |
+
SideNote: MMLU was a very solid 79+ .. weird, we'll dive further on this for irregularities :)
|
41 |
+
|
42 |
## Model Details
|
43 |
|
44 |
Adiestrated with UNA: Uniform Neural Alignment technique (paper going out soon).
|