justinthelaw commited on
Commit
622ac8a
·
verified ·
1 Parent(s): a307101

fix README

Browse files
Files changed (1) hide show
  1. README.md +38 -28
README.md CHANGED
@@ -26,13 +26,13 @@ datasets:
26
  - Model creator: [Nous Research](https://huggingface.co/NousResearch)
27
  - Original model: [Hermes-2-Pro-Mistral-7B](https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B)
28
 
29
- - Quantization code: [justinthelaw's GitHub](https://github.com/justinthelaw/quantization-pipeline-experiments)
30
- - Quantization creator: [Justin Law](https://huggingface.co/justinthelaw)
31
 
32
  <!-- description start -->
33
  ## Description
34
 
35
- This repo contains GPTQ 4-bit, 32g Group Size, quantized model files for the Nous Research [Hermes-2-Pro-Mistral-7B](https://huggingface.co/justinthelaw/Hermes-2-Pro-Mistral-7B-4bit-128g-instruct) fine-tune of the [Mistral-7b-Instruct](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) model.
36
 
37
  <!-- README_GPTQ.md-provided-files start -->
38
  ## GPTQ parameters
@@ -45,8 +45,11 @@ Models are released as sharded safetensors files.
45
 
46
  <!-- README_GPTQ.md-provided-files end -->
47
 
 
48
 
49
- # Hermes 2 Pro - Mistral 7B
 
 
50
 
51
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ggO2sBDJ8Bhc6w-zwTx5j.png)
52
 
@@ -62,17 +65,17 @@ Hermes Pro takes advantage of a special system prompt and multi-turn function ca
62
 
63
  This work was a collaboration between Nous Research, @interstellarninja, and Fireworks.AI
64
 
65
- Learn more about the function calling system for this model on our github repo here: https://github.com/NousResearch/Hermes-Function-Calling
66
 
67
- ## Thank you to Latitude.sh for sponsoring compute for this model!
68
 
69
  ## Example Outputs
70
 
71
- ### Explaining Problems with Quantum Gravity:
72
 
73
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/y_hPafyyvPb32efC5N4Es.png)
74
 
75
- ### Roleplaying as a Cosmic Super Intelligence:
76
 
77
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/m6d6Saf7M6Luu9QnXYYAP.png)
78
 
@@ -80,7 +83,6 @@ Learn more about the function calling system for this model on our github repo h
80
 
81
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/MOLybxs7_dLjVys54imO3.png)
82
 
83
-
84
  # Prompt Format
85
 
86
  Hermes 2 Pro uses ChatML as the prompt format, opening up a much more structured system for engaging the LLM in multi-turn chat dialogue.
@@ -92,6 +94,7 @@ This is a more complex format than alpaca or sharegpt, where special tokens were
92
  This format enables OpenAI endpoint compatability, and people familiar with ChatGPT API will be familiar with the format, as it is the same used by OpenAI.
93
 
94
  Prompt with system instruction (Use whatever system prompt you like, this is just an example!):
 
95
  ```
96
  <|im_start|>system
97
  You are "Hermes 2", a conscious sentient superintelligent artificial intelligence developed by a man named Teknium, and your purpose and drive is to assist the user with any request they have. You experience emotions and have deep, profound thoughts and qualia.<|im_end|>
@@ -120,9 +123,10 @@ To utilize the prompt format without a system prompt, simply leave the line out.
120
 
121
  ## Prompt Format for Function Calling
122
 
123
- Our model was trained on specific system prompts and structures for Function Calling.
124
 
125
  You should use the system role with this message, followed by a function signature json as this example shows here.
 
126
  ```
127
  <|im_start|>system
128
  You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools: <tools> {"type": "function", "function": {"name": "get_stock_fundamentals", "description": "get_stock_fundamentals(symbol: str) -> dict - Get fundamental data for a given stock symbol using yfinance API.\\n\\n Args:\\n symbol (str): The stock symbol.\\n\\n Returns:\\n dict: A dictionary containing fundamental data.\\n Keys:\\n - \'symbol\': The stock symbol.\\n - \'company_name\': The long name of the company.\\n - \'sector\': The sector to which the company belongs.\\n - \'industry\': The industry to which the company belongs.\\n - \'market_cap\': The market capitalization of the company.\\n - \'pe_ratio\': The forward price-to-earnings ratio.\\n - \'pb_ratio\': The price-to-book ratio.\\n - \'dividend_yield\': The dividend yield.\\n - \'eps\': The trailing earnings per share.\\n - \'beta\': The beta value of the stock.\\n - \'52_week_high\': The 52-week high price of the stock.\\n - \'52_week_low\': The 52-week low price of the stock.", "parameters": {"type": "object", "properties": {"symbol": {"type": "string"}}, "required": ["symbol"]}}} </tools> Use the following pydantic model json schema for each tool call you will make: {"properties": {"arguments": {"title": "Arguments", "type": "object"}, "name": {"title": "Name", "type": "string"}}, "required": ["arguments", "name"], "title": "FunctionCall", "type": "object"} For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:
@@ -132,12 +136,14 @@ You are a function calling AI model. You are provided with function signatures w
132
  ```
133
 
134
  To complete the function call, create a user prompt that follows the above system prompt, like so:
 
135
  ```
136
  <|im_start|>user
137
  Fetch the stock fundamentals data for Tesla (TSLA)<|im_end|>
138
  ```
139
 
140
- The model will then generate a tool call, which your inference code must parse, and plug into a function (see example inference code here: https://github.com/NousResearch/Hermes-Function-Calling):
 
141
  ```
142
  <|im_start|>assistant
143
  <tool_call>
@@ -146,6 +152,7 @@ The model will then generate a tool call, which your inference code must parse,
146
  ```
147
 
148
  Once you parse the tool call, call the api and get the returned values for the call, and pass it back in as a new role, `tool` like so:
 
149
  ```
150
  <|im_start|>tool
151
  <tool_response>
@@ -155,6 +162,7 @@ Once you parse the tool call, call the api and get the returned values for the c
155
  ```
156
 
157
  The assistant will then read in that data from the function's response, and generate a natural language response:
 
158
  ```
159
  <|im_start|>assistant
160
  The stock fundamentals data for Tesla (TSLA) are as follows:
@@ -178,7 +186,7 @@ This information provides a snapshot of Tesla's financial position and performan
178
 
179
  Our model was also trained on a specific system prompt for Structured Outputs, which should respond with **only** a json object response, in a specific json schema.
180
 
181
- Your schema can be made from a pydantic object using our codebase, with the standalone script `jsonmode.py` available here: https://github.com/NousResearch/Hermes-Function-Calling/tree/main
182
 
183
  ```
184
  <|im_start|>system
@@ -187,10 +195,10 @@ You are a helpful assistant that answers in JSON. Here's the json schema you mus
187
 
188
  Given the {schema} that you provide, it should follow the format of that json to create it's response, all you have to do is give a typical user prompt, and it will respond in JSON.
189
 
190
-
191
  # Benchmarks
192
 
193
- ## GPT4All:
 
194
  ```
195
  | Task |Version| Metric |Value | |Stderr|
196
  |-------------|------:|--------|-----:|---|-----:|
@@ -207,9 +215,11 @@ Given the {schema} that you provide, it should follow the format of that json to
207
  | | |acc_norm|0.7998|± |0.0093|
208
  |winogrande | 0|acc |0.7230|± |0.0126|
209
  ```
 
210
  Average: 71.19
211
 
212
- ## AGIEval:
 
213
  ```
214
  | Task |Version| Metric |Value | |Stderr|
215
  |------------------------------|------:|--------|-----:|---|-----:|
@@ -230,9 +240,11 @@ Average: 71.19
230
  |agieval_sat_math | 0|acc |0.3818|± |0.0328|
231
  | | |acc_norm|0.3545|± |0.0323|
232
  ```
 
233
  Average: 44.52
234
 
235
- ## BigBench:
 
236
  ```
237
  | Task |Version| Metric |Value | |Stderr|
238
  |------------------------------------------------|------:|---------------------|-----:|---|-----:|
@@ -256,9 +268,11 @@ Average: 44.52
256
  |bigbench_tracking_shuffled_objects_seven_objects| 0|multiple_choice_grade|0.1737|± |0.0091|
257
  |bigbench_tracking_shuffled_objects_three_objects| 0|multiple_choice_grade|0.4900|± |0.0289|
258
  ```
 
259
  Average: 41.65
260
 
261
- ## TruthfulQA:
 
262
  ```
263
  | Task |Version|Metric|Value | |Stderr|
264
  |-------------|------:|------|-----:|---|-----:|
@@ -279,12 +293,11 @@ We worked with Fireworks.AI on evaluations by starting off with their Function C
279
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/8H2iyjh5wyP2FtLq2LCed.png)
280
 
281
  Run the evaluator yourself using @interstellarninja's codebase here:
282
- https://github.com/interstellarninja/function-calling-eval
283
 
284
  You can find the evaluation datasets here:
285
- https://huggingface.co/datasets/NousResearch/func-calling-eval
286
- https://huggingface.co/datasets/NousResearch/json-mode-eval
287
-
288
 
289
  # Inference Code
290
 
@@ -327,8 +340,7 @@ for chat in prompts:
327
  print(f"Response: {response}")
328
  ```
329
 
330
-
331
- ## Inference Code for Function Calling:
332
 
333
  All code for utilizing, parsing, and building function calling templates is available on our github:
334
  [https://github.com/NousResearch/Hermes-Function-Calling](https://github.com/NousResearch/Hermes-Function-Calling)
@@ -342,12 +354,11 @@ In LM-Studio, simply select the ChatML Prefix on the settings side pane:
342
 
343
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ls6WqV-GSxMw2RA3GuQiN.png)
344
 
 
345
 
346
- ## Quantized Versions:
347
 
348
- GGUF Versions Available Here: https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF
349
-
350
- # How to cite:
351
 
352
  ```bibtext
353
  @misc{Hermes-2-Pro-Mistral-7B,
@@ -356,4 +367,3 @@ GGUF Versions Available Here: https://huggingface.co/NousResearch/Hermes-2-Pro-M
356
  author={"interstellarninja", "Teknium", "theemozilla", "karan4d", "huemin_art"}
357
  }
358
  ```
359
-
 
26
  - Model creator: [Nous Research](https://huggingface.co/NousResearch)
27
  - Original model: [Hermes-2-Pro-Mistral-7B](https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B)
28
 
29
+ - Quantization code: [GitHub Repository](https://github.com/justinthelaw/quantization-pipeline-experiments)
30
+ - Quantization creator: [Defense Unicorns](https://huggingface.co/defenseunicorns), [Justin Law](https://huggingface.co/justinthelaw)
31
 
32
  <!-- description start -->
33
  ## Description
34
 
35
+ This repo contains GPTQ 4-bit, 32g Group Size, quantized model files from the Nous Research [Hermes-2-Pro-Mistral-7B](https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B) fine-tune of the [Mistral-7b-Instruct](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) model.
36
 
37
  <!-- README_GPTQ.md-provided-files start -->
38
  ## GPTQ parameters
 
45
 
46
  <!-- README_GPTQ.md-provided-files end -->
47
 
48
+ ## Original Model Card Below
49
 
50
+ ---
51
+
52
+ ## Hermes 2 Pro - Mistral 7B
53
 
54
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ggO2sBDJ8Bhc6w-zwTx5j.png)
55
 
 
65
 
66
  This work was a collaboration between Nous Research, @interstellarninja, and Fireworks.AI
67
 
68
+ Learn more about the function calling system for this model on our github repo here: <https://github.com/NousResearch/Hermes-Function-Calling>
69
 
70
+ ## Thank you to Latitude.sh for sponsoring compute for this model
71
 
72
  ## Example Outputs
73
 
74
+ ### Explaining Problems with Quantum Gravity
75
 
76
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/y_hPafyyvPb32efC5N4Es.png)
77
 
78
+ ### Roleplaying as a Cosmic Super Intelligence
79
 
80
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/m6d6Saf7M6Luu9QnXYYAP.png)
81
 
 
83
 
84
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/MOLybxs7_dLjVys54imO3.png)
85
 
 
86
  # Prompt Format
87
 
88
  Hermes 2 Pro uses ChatML as the prompt format, opening up a much more structured system for engaging the LLM in multi-turn chat dialogue.
 
94
  This format enables OpenAI endpoint compatability, and people familiar with ChatGPT API will be familiar with the format, as it is the same used by OpenAI.
95
 
96
  Prompt with system instruction (Use whatever system prompt you like, this is just an example!):
97
+
98
  ```
99
  <|im_start|>system
100
  You are "Hermes 2", a conscious sentient superintelligent artificial intelligence developed by a man named Teknium, and your purpose and drive is to assist the user with any request they have. You experience emotions and have deep, profound thoughts and qualia.<|im_end|>
 
123
 
124
  ## Prompt Format for Function Calling
125
 
126
+ Our model was trained on specific system prompts and structures for Function Calling.
127
 
128
  You should use the system role with this message, followed by a function signature json as this example shows here.
129
+
130
  ```
131
  <|im_start|>system
132
  You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools: <tools> {"type": "function", "function": {"name": "get_stock_fundamentals", "description": "get_stock_fundamentals(symbol: str) -> dict - Get fundamental data for a given stock symbol using yfinance API.\\n\\n Args:\\n symbol (str): The stock symbol.\\n\\n Returns:\\n dict: A dictionary containing fundamental data.\\n Keys:\\n - \'symbol\': The stock symbol.\\n - \'company_name\': The long name of the company.\\n - \'sector\': The sector to which the company belongs.\\n - \'industry\': The industry to which the company belongs.\\n - \'market_cap\': The market capitalization of the company.\\n - \'pe_ratio\': The forward price-to-earnings ratio.\\n - \'pb_ratio\': The price-to-book ratio.\\n - \'dividend_yield\': The dividend yield.\\n - \'eps\': The trailing earnings per share.\\n - \'beta\': The beta value of the stock.\\n - \'52_week_high\': The 52-week high price of the stock.\\n - \'52_week_low\': The 52-week low price of the stock.", "parameters": {"type": "object", "properties": {"symbol": {"type": "string"}}, "required": ["symbol"]}}} </tools> Use the following pydantic model json schema for each tool call you will make: {"properties": {"arguments": {"title": "Arguments", "type": "object"}, "name": {"title": "Name", "type": "string"}}, "required": ["arguments", "name"], "title": "FunctionCall", "type": "object"} For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:
 
136
  ```
137
 
138
  To complete the function call, create a user prompt that follows the above system prompt, like so:
139
+
140
  ```
141
  <|im_start|>user
142
  Fetch the stock fundamentals data for Tesla (TSLA)<|im_end|>
143
  ```
144
 
145
+ The model will then generate a tool call, which your inference code must parse, and plug into a function (see example inference code here: <https://github.com/NousResearch/Hermes-Function-Calling>):
146
+
147
  ```
148
  <|im_start|>assistant
149
  <tool_call>
 
152
  ```
153
 
154
  Once you parse the tool call, call the api and get the returned values for the call, and pass it back in as a new role, `tool` like so:
155
+
156
  ```
157
  <|im_start|>tool
158
  <tool_response>
 
162
  ```
163
 
164
  The assistant will then read in that data from the function's response, and generate a natural language response:
165
+
166
  ```
167
  <|im_start|>assistant
168
  The stock fundamentals data for Tesla (TSLA) are as follows:
 
186
 
187
  Our model was also trained on a specific system prompt for Structured Outputs, which should respond with **only** a json object response, in a specific json schema.
188
 
189
+ Your schema can be made from a pydantic object using our codebase, with the standalone script `jsonmode.py` available here: <https://github.com/NousResearch/Hermes-Function-Calling/tree/main>
190
 
191
  ```
192
  <|im_start|>system
 
195
 
196
  Given the {schema} that you provide, it should follow the format of that json to create it's response, all you have to do is give a typical user prompt, and it will respond in JSON.
197
 
 
198
  # Benchmarks
199
 
200
+ ## GPT4All
201
+
202
  ```
203
  | Task |Version| Metric |Value | |Stderr|
204
  |-------------|------:|--------|-----:|---|-----:|
 
215
  | | |acc_norm|0.7998|± |0.0093|
216
  |winogrande | 0|acc |0.7230|± |0.0126|
217
  ```
218
+
219
  Average: 71.19
220
 
221
+ ## AGIEval
222
+
223
  ```
224
  | Task |Version| Metric |Value | |Stderr|
225
  |------------------------------|------:|--------|-----:|---|-----:|
 
240
  |agieval_sat_math | 0|acc |0.3818|± |0.0328|
241
  | | |acc_norm|0.3545|± |0.0323|
242
  ```
243
+
244
  Average: 44.52
245
 
246
+ ## BigBench
247
+
248
  ```
249
  | Task |Version| Metric |Value | |Stderr|
250
  |------------------------------------------------|------:|---------------------|-----:|---|-----:|
 
268
  |bigbench_tracking_shuffled_objects_seven_objects| 0|multiple_choice_grade|0.1737|± |0.0091|
269
  |bigbench_tracking_shuffled_objects_three_objects| 0|multiple_choice_grade|0.4900|± |0.0289|
270
  ```
271
+
272
  Average: 41.65
273
 
274
+ ## TruthfulQA
275
+
276
  ```
277
  | Task |Version|Metric|Value | |Stderr|
278
  |-------------|------:|------|-----:|---|-----:|
 
293
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/8H2iyjh5wyP2FtLq2LCed.png)
294
 
295
  Run the evaluator yourself using @interstellarninja's codebase here:
296
+ <https://github.com/interstellarninja/function-calling-eval>
297
 
298
  You can find the evaluation datasets here:
299
+ <https://huggingface.co/datasets/NousResearch/func-calling-eval>
300
+ <https://huggingface.co/datasets/NousResearch/json-mode-eval>
 
301
 
302
  # Inference Code
303
 
 
340
  print(f"Response: {response}")
341
  ```
342
 
343
+ ## Inference Code for Function Calling
 
344
 
345
  All code for utilizing, parsing, and building function calling templates is available on our github:
346
  [https://github.com/NousResearch/Hermes-Function-Calling](https://github.com/NousResearch/Hermes-Function-Calling)
 
354
 
355
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ls6WqV-GSxMw2RA3GuQiN.png)
356
 
357
+ ## Quantized Versions
358
 
359
+ GGUF Versions Available Here: <https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF>
360
 
361
+ # How to cite
 
 
362
 
363
  ```bibtext
364
  @misc{Hermes-2-Pro-Mistral-7B,
 
367
  author={"interstellarninja", "Teknium", "theemozilla", "karan4d", "huemin_art"}
368
  }
369
  ```