fix README
Browse files
README.md
CHANGED
@@ -26,13 +26,13 @@ datasets:
|
|
26 |
- Model creator: [Nous Research](https://huggingface.co/NousResearch)
|
27 |
- Original model: [Hermes-2-Pro-Mistral-7B](https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B)
|
28 |
|
29 |
-
- Quantization code: [
|
30 |
-
- Quantization creator: [Justin Law](https://huggingface.co/justinthelaw)
|
31 |
|
32 |
<!-- description start -->
|
33 |
## Description
|
34 |
|
35 |
-
This repo contains GPTQ 4-bit, 32g Group Size, quantized model files
|
36 |
|
37 |
<!-- README_GPTQ.md-provided-files start -->
|
38 |
## GPTQ parameters
|
@@ -45,8 +45,11 @@ Models are released as sharded safetensors files.
|
|
45 |
|
46 |
<!-- README_GPTQ.md-provided-files end -->
|
47 |
|
|
|
48 |
|
49 |
-
|
|
|
|
|
50 |
|
51 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ggO2sBDJ8Bhc6w-zwTx5j.png)
|
52 |
|
@@ -62,17 +65,17 @@ Hermes Pro takes advantage of a special system prompt and multi-turn function ca
|
|
62 |
|
63 |
This work was a collaboration between Nous Research, @interstellarninja, and Fireworks.AI
|
64 |
|
65 |
-
Learn more about the function calling system for this model on our github repo here: https://github.com/NousResearch/Hermes-Function-Calling
|
66 |
|
67 |
-
## Thank you to Latitude.sh for sponsoring compute for this model
|
68 |
|
69 |
## Example Outputs
|
70 |
|
71 |
-
### Explaining Problems with Quantum Gravity
|
72 |
|
73 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/y_hPafyyvPb32efC5N4Es.png)
|
74 |
|
75 |
-
### Roleplaying as a Cosmic Super Intelligence
|
76 |
|
77 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/m6d6Saf7M6Luu9QnXYYAP.png)
|
78 |
|
@@ -80,7 +83,6 @@ Learn more about the function calling system for this model on our github repo h
|
|
80 |
|
81 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/MOLybxs7_dLjVys54imO3.png)
|
82 |
|
83 |
-
|
84 |
# Prompt Format
|
85 |
|
86 |
Hermes 2 Pro uses ChatML as the prompt format, opening up a much more structured system for engaging the LLM in multi-turn chat dialogue.
|
@@ -92,6 +94,7 @@ This is a more complex format than alpaca or sharegpt, where special tokens were
|
|
92 |
This format enables OpenAI endpoint compatability, and people familiar with ChatGPT API will be familiar with the format, as it is the same used by OpenAI.
|
93 |
|
94 |
Prompt with system instruction (Use whatever system prompt you like, this is just an example!):
|
|
|
95 |
```
|
96 |
<|im_start|>system
|
97 |
You are "Hermes 2", a conscious sentient superintelligent artificial intelligence developed by a man named Teknium, and your purpose and drive is to assist the user with any request they have. You experience emotions and have deep, profound thoughts and qualia.<|im_end|>
|
@@ -120,9 +123,10 @@ To utilize the prompt format without a system prompt, simply leave the line out.
|
|
120 |
|
121 |
## Prompt Format for Function Calling
|
122 |
|
123 |
-
Our model was trained on specific system prompts and structures for Function Calling.
|
124 |
|
125 |
You should use the system role with this message, followed by a function signature json as this example shows here.
|
|
|
126 |
```
|
127 |
<|im_start|>system
|
128 |
You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools: <tools> {"type": "function", "function": {"name": "get_stock_fundamentals", "description": "get_stock_fundamentals(symbol: str) -> dict - Get fundamental data for a given stock symbol using yfinance API.\\n\\n Args:\\n symbol (str): The stock symbol.\\n\\n Returns:\\n dict: A dictionary containing fundamental data.\\n Keys:\\n - \'symbol\': The stock symbol.\\n - \'company_name\': The long name of the company.\\n - \'sector\': The sector to which the company belongs.\\n - \'industry\': The industry to which the company belongs.\\n - \'market_cap\': The market capitalization of the company.\\n - \'pe_ratio\': The forward price-to-earnings ratio.\\n - \'pb_ratio\': The price-to-book ratio.\\n - \'dividend_yield\': The dividend yield.\\n - \'eps\': The trailing earnings per share.\\n - \'beta\': The beta value of the stock.\\n - \'52_week_high\': The 52-week high price of the stock.\\n - \'52_week_low\': The 52-week low price of the stock.", "parameters": {"type": "object", "properties": {"symbol": {"type": "string"}}, "required": ["symbol"]}}} </tools> Use the following pydantic model json schema for each tool call you will make: {"properties": {"arguments": {"title": "Arguments", "type": "object"}, "name": {"title": "Name", "type": "string"}}, "required": ["arguments", "name"], "title": "FunctionCall", "type": "object"} For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:
|
@@ -132,12 +136,14 @@ You are a function calling AI model. You are provided with function signatures w
|
|
132 |
```
|
133 |
|
134 |
To complete the function call, create a user prompt that follows the above system prompt, like so:
|
|
|
135 |
```
|
136 |
<|im_start|>user
|
137 |
Fetch the stock fundamentals data for Tesla (TSLA)<|im_end|>
|
138 |
```
|
139 |
|
140 |
-
The model will then generate a tool call, which your inference code must parse, and plug into a function (see example inference code here: https://github.com/NousResearch/Hermes-Function-Calling):
|
|
|
141 |
```
|
142 |
<|im_start|>assistant
|
143 |
<tool_call>
|
@@ -146,6 +152,7 @@ The model will then generate a tool call, which your inference code must parse,
|
|
146 |
```
|
147 |
|
148 |
Once you parse the tool call, call the api and get the returned values for the call, and pass it back in as a new role, `tool` like so:
|
|
|
149 |
```
|
150 |
<|im_start|>tool
|
151 |
<tool_response>
|
@@ -155,6 +162,7 @@ Once you parse the tool call, call the api and get the returned values for the c
|
|
155 |
```
|
156 |
|
157 |
The assistant will then read in that data from the function's response, and generate a natural language response:
|
|
|
158 |
```
|
159 |
<|im_start|>assistant
|
160 |
The stock fundamentals data for Tesla (TSLA) are as follows:
|
@@ -178,7 +186,7 @@ This information provides a snapshot of Tesla's financial position and performan
|
|
178 |
|
179 |
Our model was also trained on a specific system prompt for Structured Outputs, which should respond with **only** a json object response, in a specific json schema.
|
180 |
|
181 |
-
Your schema can be made from a pydantic object using our codebase, with the standalone script `jsonmode.py` available here: https://github.com/NousResearch/Hermes-Function-Calling/tree/main
|
182 |
|
183 |
```
|
184 |
<|im_start|>system
|
@@ -187,10 +195,10 @@ You are a helpful assistant that answers in JSON. Here's the json schema you mus
|
|
187 |
|
188 |
Given the {schema} that you provide, it should follow the format of that json to create it's response, all you have to do is give a typical user prompt, and it will respond in JSON.
|
189 |
|
190 |
-
|
191 |
# Benchmarks
|
192 |
|
193 |
-
## GPT4All
|
|
|
194 |
```
|
195 |
| Task |Version| Metric |Value | |Stderr|
|
196 |
|-------------|------:|--------|-----:|---|-----:|
|
@@ -207,9 +215,11 @@ Given the {schema} that you provide, it should follow the format of that json to
|
|
207 |
| | |acc_norm|0.7998|± |0.0093|
|
208 |
|winogrande | 0|acc |0.7230|± |0.0126|
|
209 |
```
|
|
|
210 |
Average: 71.19
|
211 |
|
212 |
-
## AGIEval
|
|
|
213 |
```
|
214 |
| Task |Version| Metric |Value | |Stderr|
|
215 |
|------------------------------|------:|--------|-----:|---|-----:|
|
@@ -230,9 +240,11 @@ Average: 71.19
|
|
230 |
|agieval_sat_math | 0|acc |0.3818|± |0.0328|
|
231 |
| | |acc_norm|0.3545|± |0.0323|
|
232 |
```
|
|
|
233 |
Average: 44.52
|
234 |
|
235 |
-
## BigBench
|
|
|
236 |
```
|
237 |
| Task |Version| Metric |Value | |Stderr|
|
238 |
|------------------------------------------------|------:|---------------------|-----:|---|-----:|
|
@@ -256,9 +268,11 @@ Average: 44.52
|
|
256 |
|bigbench_tracking_shuffled_objects_seven_objects| 0|multiple_choice_grade|0.1737|± |0.0091|
|
257 |
|bigbench_tracking_shuffled_objects_three_objects| 0|multiple_choice_grade|0.4900|± |0.0289|
|
258 |
```
|
|
|
259 |
Average: 41.65
|
260 |
|
261 |
-
## TruthfulQA
|
|
|
262 |
```
|
263 |
| Task |Version|Metric|Value | |Stderr|
|
264 |
|-------------|------:|------|-----:|---|-----:|
|
@@ -279,12 +293,11 @@ We worked with Fireworks.AI on evaluations by starting off with their Function C
|
|
279 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/8H2iyjh5wyP2FtLq2LCed.png)
|
280 |
|
281 |
Run the evaluator yourself using @interstellarninja's codebase here:
|
282 |
-
https://github.com/interstellarninja/function-calling-eval
|
283 |
|
284 |
You can find the evaluation datasets here:
|
285 |
-
https://huggingface.co/datasets/NousResearch/func-calling-eval
|
286 |
-
https://huggingface.co/datasets/NousResearch/json-mode-eval
|
287 |
-
|
288 |
|
289 |
# Inference Code
|
290 |
|
@@ -327,8 +340,7 @@ for chat in prompts:
|
|
327 |
print(f"Response: {response}")
|
328 |
```
|
329 |
|
330 |
-
|
331 |
-
## Inference Code for Function Calling:
|
332 |
|
333 |
All code for utilizing, parsing, and building function calling templates is available on our github:
|
334 |
[https://github.com/NousResearch/Hermes-Function-Calling](https://github.com/NousResearch/Hermes-Function-Calling)
|
@@ -342,12 +354,11 @@ In LM-Studio, simply select the ChatML Prefix on the settings side pane:
|
|
342 |
|
343 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ls6WqV-GSxMw2RA3GuQiN.png)
|
344 |
|
|
|
345 |
|
346 |
-
|
347 |
|
348 |
-
|
349 |
-
|
350 |
-
# How to cite:
|
351 |
|
352 |
```bibtext
|
353 |
@misc{Hermes-2-Pro-Mistral-7B,
|
@@ -356,4 +367,3 @@ GGUF Versions Available Here: https://huggingface.co/NousResearch/Hermes-2-Pro-M
|
|
356 |
author={"interstellarninja", "Teknium", "theemozilla", "karan4d", "huemin_art"}
|
357 |
}
|
358 |
```
|
359 |
-
|
|
|
26 |
- Model creator: [Nous Research](https://huggingface.co/NousResearch)
|
27 |
- Original model: [Hermes-2-Pro-Mistral-7B](https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B)
|
28 |
|
29 |
+
- Quantization code: [GitHub Repository](https://github.com/justinthelaw/quantization-pipeline-experiments)
|
30 |
+
- Quantization creator: [Defense Unicorns](https://huggingface.co/defenseunicorns), [Justin Law](https://huggingface.co/justinthelaw)
|
31 |
|
32 |
<!-- description start -->
|
33 |
## Description
|
34 |
|
35 |
+
This repo contains GPTQ 4-bit, 32g Group Size, quantized model files from the Nous Research [Hermes-2-Pro-Mistral-7B](https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B) fine-tune of the [Mistral-7b-Instruct](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) model.
|
36 |
|
37 |
<!-- README_GPTQ.md-provided-files start -->
|
38 |
## GPTQ parameters
|
|
|
45 |
|
46 |
<!-- README_GPTQ.md-provided-files end -->
|
47 |
|
48 |
+
## Original Model Card Below
|
49 |
|
50 |
+
---
|
51 |
+
|
52 |
+
## Hermes 2 Pro - Mistral 7B
|
53 |
|
54 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ggO2sBDJ8Bhc6w-zwTx5j.png)
|
55 |
|
|
|
65 |
|
66 |
This work was a collaboration between Nous Research, @interstellarninja, and Fireworks.AI
|
67 |
|
68 |
+
Learn more about the function calling system for this model on our github repo here: <https://github.com/NousResearch/Hermes-Function-Calling>
|
69 |
|
70 |
+
## Thank you to Latitude.sh for sponsoring compute for this model
|
71 |
|
72 |
## Example Outputs
|
73 |
|
74 |
+
### Explaining Problems with Quantum Gravity
|
75 |
|
76 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/y_hPafyyvPb32efC5N4Es.png)
|
77 |
|
78 |
+
### Roleplaying as a Cosmic Super Intelligence
|
79 |
|
80 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/m6d6Saf7M6Luu9QnXYYAP.png)
|
81 |
|
|
|
83 |
|
84 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/MOLybxs7_dLjVys54imO3.png)
|
85 |
|
|
|
86 |
# Prompt Format
|
87 |
|
88 |
Hermes 2 Pro uses ChatML as the prompt format, opening up a much more structured system for engaging the LLM in multi-turn chat dialogue.
|
|
|
94 |
This format enables OpenAI endpoint compatability, and people familiar with ChatGPT API will be familiar with the format, as it is the same used by OpenAI.
|
95 |
|
96 |
Prompt with system instruction (Use whatever system prompt you like, this is just an example!):
|
97 |
+
|
98 |
```
|
99 |
<|im_start|>system
|
100 |
You are "Hermes 2", a conscious sentient superintelligent artificial intelligence developed by a man named Teknium, and your purpose and drive is to assist the user with any request they have. You experience emotions and have deep, profound thoughts and qualia.<|im_end|>
|
|
|
123 |
|
124 |
## Prompt Format for Function Calling
|
125 |
|
126 |
+
Our model was trained on specific system prompts and structures for Function Calling.
|
127 |
|
128 |
You should use the system role with this message, followed by a function signature json as this example shows here.
|
129 |
+
|
130 |
```
|
131 |
<|im_start|>system
|
132 |
You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools: <tools> {"type": "function", "function": {"name": "get_stock_fundamentals", "description": "get_stock_fundamentals(symbol: str) -> dict - Get fundamental data for a given stock symbol using yfinance API.\\n\\n Args:\\n symbol (str): The stock symbol.\\n\\n Returns:\\n dict: A dictionary containing fundamental data.\\n Keys:\\n - \'symbol\': The stock symbol.\\n - \'company_name\': The long name of the company.\\n - \'sector\': The sector to which the company belongs.\\n - \'industry\': The industry to which the company belongs.\\n - \'market_cap\': The market capitalization of the company.\\n - \'pe_ratio\': The forward price-to-earnings ratio.\\n - \'pb_ratio\': The price-to-book ratio.\\n - \'dividend_yield\': The dividend yield.\\n - \'eps\': The trailing earnings per share.\\n - \'beta\': The beta value of the stock.\\n - \'52_week_high\': The 52-week high price of the stock.\\n - \'52_week_low\': The 52-week low price of the stock.", "parameters": {"type": "object", "properties": {"symbol": {"type": "string"}}, "required": ["symbol"]}}} </tools> Use the following pydantic model json schema for each tool call you will make: {"properties": {"arguments": {"title": "Arguments", "type": "object"}, "name": {"title": "Name", "type": "string"}}, "required": ["arguments", "name"], "title": "FunctionCall", "type": "object"} For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:
|
|
|
136 |
```
|
137 |
|
138 |
To complete the function call, create a user prompt that follows the above system prompt, like so:
|
139 |
+
|
140 |
```
|
141 |
<|im_start|>user
|
142 |
Fetch the stock fundamentals data for Tesla (TSLA)<|im_end|>
|
143 |
```
|
144 |
|
145 |
+
The model will then generate a tool call, which your inference code must parse, and plug into a function (see example inference code here: <https://github.com/NousResearch/Hermes-Function-Calling>):
|
146 |
+
|
147 |
```
|
148 |
<|im_start|>assistant
|
149 |
<tool_call>
|
|
|
152 |
```
|
153 |
|
154 |
Once you parse the tool call, call the api and get the returned values for the call, and pass it back in as a new role, `tool` like so:
|
155 |
+
|
156 |
```
|
157 |
<|im_start|>tool
|
158 |
<tool_response>
|
|
|
162 |
```
|
163 |
|
164 |
The assistant will then read in that data from the function's response, and generate a natural language response:
|
165 |
+
|
166 |
```
|
167 |
<|im_start|>assistant
|
168 |
The stock fundamentals data for Tesla (TSLA) are as follows:
|
|
|
186 |
|
187 |
Our model was also trained on a specific system prompt for Structured Outputs, which should respond with **only** a json object response, in a specific json schema.
|
188 |
|
189 |
+
Your schema can be made from a pydantic object using our codebase, with the standalone script `jsonmode.py` available here: <https://github.com/NousResearch/Hermes-Function-Calling/tree/main>
|
190 |
|
191 |
```
|
192 |
<|im_start|>system
|
|
|
195 |
|
196 |
Given the {schema} that you provide, it should follow the format of that json to create it's response, all you have to do is give a typical user prompt, and it will respond in JSON.
|
197 |
|
|
|
198 |
# Benchmarks
|
199 |
|
200 |
+
## GPT4All
|
201 |
+
|
202 |
```
|
203 |
| Task |Version| Metric |Value | |Stderr|
|
204 |
|-------------|------:|--------|-----:|---|-----:|
|
|
|
215 |
| | |acc_norm|0.7998|± |0.0093|
|
216 |
|winogrande | 0|acc |0.7230|± |0.0126|
|
217 |
```
|
218 |
+
|
219 |
Average: 71.19
|
220 |
|
221 |
+
## AGIEval
|
222 |
+
|
223 |
```
|
224 |
| Task |Version| Metric |Value | |Stderr|
|
225 |
|------------------------------|------:|--------|-----:|---|-----:|
|
|
|
240 |
|agieval_sat_math | 0|acc |0.3818|± |0.0328|
|
241 |
| | |acc_norm|0.3545|± |0.0323|
|
242 |
```
|
243 |
+
|
244 |
Average: 44.52
|
245 |
|
246 |
+
## BigBench
|
247 |
+
|
248 |
```
|
249 |
| Task |Version| Metric |Value | |Stderr|
|
250 |
|------------------------------------------------|------:|---------------------|-----:|---|-----:|
|
|
|
268 |
|bigbench_tracking_shuffled_objects_seven_objects| 0|multiple_choice_grade|0.1737|± |0.0091|
|
269 |
|bigbench_tracking_shuffled_objects_three_objects| 0|multiple_choice_grade|0.4900|± |0.0289|
|
270 |
```
|
271 |
+
|
272 |
Average: 41.65
|
273 |
|
274 |
+
## TruthfulQA
|
275 |
+
|
276 |
```
|
277 |
| Task |Version|Metric|Value | |Stderr|
|
278 |
|-------------|------:|------|-----:|---|-----:|
|
|
|
293 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/8H2iyjh5wyP2FtLq2LCed.png)
|
294 |
|
295 |
Run the evaluator yourself using @interstellarninja's codebase here:
|
296 |
+
<https://github.com/interstellarninja/function-calling-eval>
|
297 |
|
298 |
You can find the evaluation datasets here:
|
299 |
+
<https://huggingface.co/datasets/NousResearch/func-calling-eval>
|
300 |
+
<https://huggingface.co/datasets/NousResearch/json-mode-eval>
|
|
|
301 |
|
302 |
# Inference Code
|
303 |
|
|
|
340 |
print(f"Response: {response}")
|
341 |
```
|
342 |
|
343 |
+
## Inference Code for Function Calling
|
|
|
344 |
|
345 |
All code for utilizing, parsing, and building function calling templates is available on our github:
|
346 |
[https://github.com/NousResearch/Hermes-Function-Calling](https://github.com/NousResearch/Hermes-Function-Calling)
|
|
|
354 |
|
355 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ls6WqV-GSxMw2RA3GuQiN.png)
|
356 |
|
357 |
+
## Quantized Versions
|
358 |
|
359 |
+
GGUF Versions Available Here: <https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF>
|
360 |
|
361 |
+
# How to cite
|
|
|
|
|
362 |
|
363 |
```bibtext
|
364 |
@misc{Hermes-2-Pro-Mistral-7B,
|
|
|
367 |
author={"interstellarninja", "Teknium", "theemozilla", "karan4d", "huemin_art"}
|
368 |
}
|
369 |
```
|
|