Text Generation
Safetensors
qwen2
chat
conversational
Eval Results
Files changed (1) hide show
  1. README.md +126 -18
README.md CHANGED
@@ -1,25 +1,120 @@
1
  ---
 
 
 
 
 
 
 
 
 
 
2
  license: other
3
- license_name: tongyi-qianwen
4
- license_link: https://huggingface.co/anthracite-org/magnum-v2-72b/blob/main/LICENSE
5
  base_model: Qwen/Qwen2-72B-Instruct
6
  datasets:
7
- - Doctor-Shotgun/C2-Stheno
8
- - anthracite-org/kalo-opus-instruct-22k-no-refusal
9
- - anthracite-org/nopm_claude_writing_fixed
10
- language:
11
- - en
12
- - fr
13
- - de
14
- - es
15
- - it
16
- - pt
17
- - ru
18
- - zh
19
- - ja
20
  pipeline_tag: text-generation
21
- tags:
22
- - chat
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
  ---
24
 
25
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6491e00e057b0928b3e07b75/u8B-5bEeroN549uxUIisV.png)
@@ -58,4 +153,17 @@ Sample Packing was done for 16k tokens rather than the 8k tokens used in our pre
58
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
59
 
60
  ## Safety
61
- ...
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ - fr
5
+ - de
6
+ - es
7
+ - it
8
+ - pt
9
+ - ru
10
+ - zh
11
+ - ja
12
  license: other
13
+ tags:
14
+ - chat
15
  base_model: Qwen/Qwen2-72B-Instruct
16
  datasets:
17
+ - Doctor-Shotgun/C2-Stheno
18
+ - anthracite-org/kalo-opus-instruct-22k-no-refusal
19
+ - anthracite-org/nopm_claude_writing_fixed
20
+ license_name: tongyi-qianwen
21
+ license_link: https://huggingface.co/anthracite-org/magnum-v2-72b/blob/main/LICENSE
 
 
 
 
 
 
 
 
22
  pipeline_tag: text-generation
23
+ model-index:
24
+ - name: magnum-v2-72b
25
+ results:
26
+ - task:
27
+ type: text-generation
28
+ name: Text Generation
29
+ dataset:
30
+ name: IFEval (0-Shot)
31
+ type: HuggingFaceH4/ifeval
32
+ args:
33
+ num_few_shot: 0
34
+ metrics:
35
+ - type: inst_level_strict_acc and prompt_level_strict_acc
36
+ value: 75.6
37
+ name: strict accuracy
38
+ source:
39
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v2-72b
40
+ name: Open LLM Leaderboard
41
+ - task:
42
+ type: text-generation
43
+ name: Text Generation
44
+ dataset:
45
+ name: BBH (3-Shot)
46
+ type: BBH
47
+ args:
48
+ num_few_shot: 3
49
+ metrics:
50
+ - type: acc_norm
51
+ value: 57.85
52
+ name: normalized accuracy
53
+ source:
54
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v2-72b
55
+ name: Open LLM Leaderboard
56
+ - task:
57
+ type: text-generation
58
+ name: Text Generation
59
+ dataset:
60
+ name: MATH Lvl 5 (4-Shot)
61
+ type: hendrycks/competition_math
62
+ args:
63
+ num_few_shot: 4
64
+ metrics:
65
+ - type: exact_match
66
+ value: 31.65
67
+ name: exact match
68
+ source:
69
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v2-72b
70
+ name: Open LLM Leaderboard
71
+ - task:
72
+ type: text-generation
73
+ name: Text Generation
74
+ dataset:
75
+ name: GPQA (0-shot)
76
+ type: Idavidrein/gpqa
77
+ args:
78
+ num_few_shot: 0
79
+ metrics:
80
+ - type: acc_norm
81
+ value: 18.12
82
+ name: acc_norm
83
+ source:
84
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v2-72b
85
+ name: Open LLM Leaderboard
86
+ - task:
87
+ type: text-generation
88
+ name: Text Generation
89
+ dataset:
90
+ name: MuSR (0-shot)
91
+ type: TAUR-Lab/MuSR
92
+ args:
93
+ num_few_shot: 0
94
+ metrics:
95
+ - type: acc_norm
96
+ value: 14.18
97
+ name: acc_norm
98
+ source:
99
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v2-72b
100
+ name: Open LLM Leaderboard
101
+ - task:
102
+ type: text-generation
103
+ name: Text Generation
104
+ dataset:
105
+ name: MMLU-PRO (5-shot)
106
+ type: TIGER-Lab/MMLU-Pro
107
+ config: main
108
+ split: test
109
+ args:
110
+ num_few_shot: 5
111
+ metrics:
112
+ - type: acc
113
+ value: 49.51
114
+ name: accuracy
115
+ source:
116
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v2-72b
117
+ name: Open LLM Leaderboard
118
  ---
119
 
120
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6491e00e057b0928b3e07b75/u8B-5bEeroN549uxUIisV.png)
 
153
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
154
 
155
  ## Safety
156
+ ...
157
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
158
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_anthracite-org__magnum-v2-72b)
159
+
160
+ | Metric |Value|
161
+ |-------------------|----:|
162
+ |Avg. |41.15|
163
+ |IFEval (0-Shot) |75.60|
164
+ |BBH (3-Shot) |57.85|
165
+ |MATH Lvl 5 (4-Shot)|31.65|
166
+ |GPQA (0-shot) |18.12|
167
+ |MuSR (0-shot) |14.18|
168
+ |MMLU-PRO (5-shot) |49.51|
169
+