bamec66557 commited on
Commit
b3e2697
1 Parent(s): 28318af

Adding Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions

Files changed (1) hide show
  1. README.md +116 -8
README.md CHANGED
@@ -1,19 +1,114 @@
1
  ---
2
- base_model:
3
- - bamec66557/mergekit-slerp-xjlmywj
4
- - bamec66557/MISCHIEVOUS-12B-Mix_0.3v
 
5
  library_name: transformers
6
  tags:
7
  - mergekit
8
  - merge
9
  - text-generation-inference
10
  - not-for-all-audiences
11
- license: apache-2.0
12
- language:
13
- - en
14
- - ko
15
  datasets:
16
  - open-llm-leaderboard/bamec66557__MISCHIEVOUS-12B-Mix_0.4v-details
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
  ---
18
 
19
  <a href="#" target="_blank"><img src="https://huggingface.co/bamec66557/MISCHIEVOUS-12B/resolve/main/00001-321918068.gif"></a>
@@ -85,4 +180,17 @@ postprocessing:
85
  kernel_size: 5 # Smoothing larger ranges with increased kernel size
86
  - operation: normalize # Normalise after merge
87
 
88
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ - ko
5
+ license: apache-2.0
6
  library_name: transformers
7
  tags:
8
  - mergekit
9
  - merge
10
  - text-generation-inference
11
  - not-for-all-audiences
12
+ base_model:
13
+ - bamec66557/mergekit-slerp-xjlmywj
14
+ - bamec66557/MISCHIEVOUS-12B-Mix_0.3v
 
15
  datasets:
16
  - open-llm-leaderboard/bamec66557__MISCHIEVOUS-12B-Mix_0.4v-details
17
+ model-index:
18
+ - name: MISCHIEVOUS-12B-Mix_0.4v
19
+ results:
20
+ - task:
21
+ type: text-generation
22
+ name: Text Generation
23
+ dataset:
24
+ name: IFEval (0-Shot)
25
+ type: HuggingFaceH4/ifeval
26
+ args:
27
+ num_few_shot: 0
28
+ metrics:
29
+ - type: inst_level_strict_acc and prompt_level_strict_acc
30
+ value: 65.08
31
+ name: strict accuracy
32
+ source:
33
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bamec66557/MISCHIEVOUS-12B-Mix_0.4v
34
+ name: Open LLM Leaderboard
35
+ - task:
36
+ type: text-generation
37
+ name: Text Generation
38
+ dataset:
39
+ name: BBH (3-Shot)
40
+ type: BBH
41
+ args:
42
+ num_few_shot: 3
43
+ metrics:
44
+ - type: acc_norm
45
+ value: 30.6
46
+ name: normalized accuracy
47
+ source:
48
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bamec66557/MISCHIEVOUS-12B-Mix_0.4v
49
+ name: Open LLM Leaderboard
50
+ - task:
51
+ type: text-generation
52
+ name: Text Generation
53
+ dataset:
54
+ name: MATH Lvl 5 (4-Shot)
55
+ type: hendrycks/competition_math
56
+ args:
57
+ num_few_shot: 4
58
+ metrics:
59
+ - type: exact_match
60
+ value: 11.48
61
+ name: exact match
62
+ source:
63
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bamec66557/MISCHIEVOUS-12B-Mix_0.4v
64
+ name: Open LLM Leaderboard
65
+ - task:
66
+ type: text-generation
67
+ name: Text Generation
68
+ dataset:
69
+ name: GPQA (0-shot)
70
+ type: Idavidrein/gpqa
71
+ args:
72
+ num_few_shot: 0
73
+ metrics:
74
+ - type: acc_norm
75
+ value: 8.95
76
+ name: acc_norm
77
+ source:
78
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bamec66557/MISCHIEVOUS-12B-Mix_0.4v
79
+ name: Open LLM Leaderboard
80
+ - task:
81
+ type: text-generation
82
+ name: Text Generation
83
+ dataset:
84
+ name: MuSR (0-shot)
85
+ type: TAUR-Lab/MuSR
86
+ args:
87
+ num_few_shot: 0
88
+ metrics:
89
+ - type: acc_norm
90
+ value: 11.97
91
+ name: acc_norm
92
+ source:
93
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bamec66557/MISCHIEVOUS-12B-Mix_0.4v
94
+ name: Open LLM Leaderboard
95
+ - task:
96
+ type: text-generation
97
+ name: Text Generation
98
+ dataset:
99
+ name: MMLU-PRO (5-shot)
100
+ type: TIGER-Lab/MMLU-Pro
101
+ config: main
102
+ split: test
103
+ args:
104
+ num_few_shot: 5
105
+ metrics:
106
+ - type: acc
107
+ value: 29.81
108
+ name: accuracy
109
+ source:
110
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=bamec66557/MISCHIEVOUS-12B-Mix_0.4v
111
+ name: Open LLM Leaderboard
112
  ---
113
 
114
  <a href="#" target="_blank"><img src="https://huggingface.co/bamec66557/MISCHIEVOUS-12B/resolve/main/00001-321918068.gif"></a>
 
180
  kernel_size: 5 # Smoothing larger ranges with increased kernel size
181
  - operation: normalize # Normalise after merge
182
 
183
+ ```
184
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
185
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/bamec66557__MISCHIEVOUS-12B-Mix_0.4v-details)
186
+
187
+ | Metric |Value|
188
+ |-------------------|----:|
189
+ |Avg. |26.31|
190
+ |IFEval (0-Shot) |65.08|
191
+ |BBH (3-Shot) |30.60|
192
+ |MATH Lvl 5 (4-Shot)|11.48|
193
+ |GPQA (0-shot) | 8.95|
194
+ |MuSR (0-shot) |11.97|
195
+ |MMLU-PRO (5-shot) |29.81|
196
+