Hugging Face's logo

CultriX
/
Qwen2.5-14B-Wernicke

Text Generation
Transformers
Safetensors
English
qwen2
mergekit
Merge
conversational
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Community
1
CultriX leaderboard-pr-bot commited on
Commit
5f19dd1
•
1 Parent(s): e1335bc

Adding Evaluation Results (#1)

Browse files

- Adding Evaluation Results (e8eae53ca51c45929c3515c8332572a5b23cee36)


Co-authored-by: Open LLM Leaderboard PR Bot <[email protected]>

Files changed (1) hide show
  1. README.md +113 -5
README.md CHANGED
@@ -1,4 +1,8 @@
1
  ---
 
 
 
 
2
  base_model:
3
  - v000000/Qwen2.5-Lumen-14B
4
  - arcee-ai/SuperNova-Medius
@@ -6,11 +10,101 @@ base_model:
6
  - rombodawg/Rombos-LLM-V2.6-Qwen-14b
7
  - Qwen/Qwen2.5-14B-Instruct
8
  - EVA-UNIT-01/EVA-Qwen2.5-14B-v0.0
9
- library_name: transformers
10
- tags:
11
- - mergekit
12
- - merge
13
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  ---
15
  # merge
16
 
@@ -45,3 +139,17 @@ base_model: Qwen/Qwen2.5-14B
45
  merge_method: model_stock
46
  dtype: bfloat16
47
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ library_name: transformers
3
+ tags:
4
+ - mergekit
5
+ - merge
6
  base_model:
7
  - v000000/Qwen2.5-Lumen-14B
8
  - arcee-ai/SuperNova-Medius
 
10
  - rombodawg/Rombos-LLM-V2.6-Qwen-14b
11
  - Qwen/Qwen2.5-14B-Instruct
12
  - EVA-UNIT-01/EVA-Qwen2.5-14B-v0.0
13
+ model-index:
14
+ - name: Qwen2.5-14B-Wernicke
15
+ results:
16
+ - task:
17
+ type: text-generation
18
+ name: Text Generation
19
+ dataset:
20
+ name: IFEval (0-Shot)
21
+ type: HuggingFaceH4/ifeval
22
+ args:
23
+ num_few_shot: 0
24
+ metrics:
25
+ - type: inst_level_strict_acc and prompt_level_strict_acc
26
+ value: 52.35
27
+ name: strict accuracy
28
+ source:
29
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=CultriX/Qwen2.5-14B-Wernicke
30
+ name: Open LLM Leaderboard
31
+ - task:
32
+ type: text-generation
33
+ name: Text Generation
34
+ dataset:
35
+ name: BBH (3-Shot)
36
+ type: BBH
37
+ args:
38
+ num_few_shot: 3
39
+ metrics:
40
+ - type: acc_norm
41
+ value: 50.64
42
+ name: normalized accuracy
43
+ source:
44
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=CultriX/Qwen2.5-14B-Wernicke
45
+ name: Open LLM Leaderboard
46
+ - task:
47
+ type: text-generation
48
+ name: Text Generation
49
+ dataset:
50
+ name: MATH Lvl 5 (4-Shot)
51
+ type: hendrycks/competition_math
52
+ args:
53
+ num_few_shot: 4
54
+ metrics:
55
+ - type: exact_match
56
+ value: 30.06
57
+ name: exact match
58
+ source:
59
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=CultriX/Qwen2.5-14B-Wernicke
60
+ name: Open LLM Leaderboard
61
+ - task:
62
+ type: text-generation
63
+ name: Text Generation
64
+ dataset:
65
+ name: GPQA (0-shot)
66
+ type: Idavidrein/gpqa
67
+ args:
68
+ num_few_shot: 0
69
+ metrics:
70
+ - type: acc_norm
71
+ value: 19.13
72
+ name: acc_norm
73
+ source:
74
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=CultriX/Qwen2.5-14B-Wernicke
75
+ name: Open LLM Leaderboard
76
+ - task:
77
+ type: text-generation
78
+ name: Text Generation
79
+ dataset:
80
+ name: MuSR (0-shot)
81
+ type: TAUR-Lab/MuSR
82
+ args:
83
+ num_few_shot: 0
84
+ metrics:
85
+ - type: acc_norm
86
+ value: 18.25
87
+ name: acc_norm
88
+ source:
89
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=CultriX/Qwen2.5-14B-Wernicke
90
+ name: Open LLM Leaderboard
91
+ - task:
92
+ type: text-generation
93
+ name: Text Generation
94
+ dataset:
95
+ name: MMLU-PRO (5-shot)
96
+ type: TIGER-Lab/MMLU-Pro
97
+ config: main
98
+ split: test
99
+ args:
100
+ num_few_shot: 5
101
+ metrics:
102
+ - type: acc
103
+ value: 49.15
104
+ name: accuracy
105
+ source:
106
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=CultriX/Qwen2.5-14B-Wernicke
107
+ name: Open LLM Leaderboard
108
  ---
109
  # merge
110
 
 
139
  merge_method: model_stock
140
  dtype: bfloat16
141
  ```
142
+
143
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
144
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_CultriX__Qwen2.5-14B-Wernicke)
145
+
146
+ | Metric |Value|
147
+ |-------------------|----:|
148
+ |Avg. |36.60|
149
+ |IFEval (0-Shot) |52.35|
150
+ |BBH (3-Shot) |50.64|
151
+ |MATH Lvl 5 (4-Shot)|30.06|
152
+ |GPQA (0-shot) |19.13|
153
+ |MuSR (0-shot) |18.25|
154
+ |MMLU-PRO (5-shot) |49.15|
155
+