Update README.md

README.md

@@ -1,185 +1,87 @@
 ---
-license: llama2
-base_model: codellama/CodeLlama-13b-Instruct-hf
-model-index:
-- name: NexusRaven-13B
-  results: []
+license: apache-2.0
+pipeline_tag: text-generation
+tags:
+- finetuned
+inference:
+  parameters:
+    temperature: 0.7
 ---
-# NexusRaven-13B: Surpassing the state-of-the-art in open-source function calling LLMs. 
 
-<p align="center">
-<a href="https://huggingface.co/Nexusflow" target="_blank">Nexusflow HF</a> - <a href="http://nexusflow.ai/blog" target="_blank">NexusRaven blog post</a> - <a href="https://huggingface.co/Nexusflow/NexusRaven-13B" target="_blank">NexusRaven-13B</a> - <a href="https://x.com/NexusflowX/status/1707470614012035561?s=20" target="_blank">NexusRaven-13B Twitter Thread</a> - <a href="https://github.com/nexusflowai/NexusRaven/" target="_blank">NexusRaven-13B Github</a> - <a href="https://huggingface.co/datasets/Nexusflow/NexusRaven_API_evaluation" target="_blank">NexusRaven API evaluation dataset</a>
-</p>
+# Model Card for Mistral-7B-Instruct-v0.1
 
-<p align="center" width="100%">
-<a><img src="NexusRaven.png" alt="NexusRaven" style="width: 40%; min-width: 300px; display: block; margin: auto;"></a>
-</p>
+The Mistral-7B-Instruct-v0.1 Large Language Model (LLM) is a instruct fine-tuned version of the [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) generative text model using a variety of publicly available conversation datasets.
 
-Table of contents
-- [NexusRaven-13B: Surpassing the state-of-the-art in open-source function calling LLMs.](#nexusraven-13b-surpassing-the-state-of-the-art-in-open-source-function-calling-llms)
-  - [Introducing NexusRaven-13B](#introducing-nexusraven-13b)
-  - [NexusRaven model usage](#nexusraven-model-usage)
-  - [Training procedure](#training-procedure)
-    - [Training hyperparameters](#training-hyperparameters)
-    - [Framework versions](#framework-versions)
-- [Limitations](#limitations)
-  - [License](#license)
-  - [References](#references)
-  - [Citation](#citation)
-  - [Contact](#contact)
+For full details of this model please read our [release blog post](https://mistral.ai/news/announcing-mistral-7b/)
 
+## Instruction format
 
-This model is a fine-tuned version of [codellama/CodeLlama-13b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-13b-Instruct-hf).
+In order to leverage instruction fine-tuning, your prompt should be surrounded by `[INST]` and `[\INST]` tokens. The very first instruction should begin with a begin of sentence id. The next instructions should not. The assistant generation will be ended by the end-of-sentence token id.
 
-## Introducing NexusRaven-13B
-NexusRaven is an open-source and commercially viable function calling LLM that surpasses the state-of-the-art in function calling capabilities. 
-
-📊 Performance Highlights: With our demonstration retrieval system, NexusRaven-13B achieves a 95% success rate in using cybersecurity tools such as CVE/CPE Search and VirusTotal, while prompting GPT-4 achieves 64%. It has significantly lower cost and faster inference speed compared to GPT-4.
-
-🔧 Generalization to the Unseen: NexusRaven-13B generalizes to tools never seen during model training, achieving a success rate comparable with GPT-3.5 in zero-shot setting, significantly outperforming all other open-source LLMs of similar sizes. 
-
-🔥 Commercially Permissive: The training of NexusRaven-13B does not involve any data generated by proprietary LLMs such as GPT-4. You have full control of the model when deployed in commercial applications.
-
-<p align="center" width="100%">
-<a><img src="Single-Attempt_Function_Calling.png" alt="NexusRaven" style="width: 80%; min-width: 300px; display: block; margin: auto;"></a>
-<a><img src="Zero-shot_Evaluation.png" alt="NexusRaven" style="width: 80%; min-width: 300px; display: block; margin: auto;"></a>
-</p>
-
-
-## NexusRaven model usage
-NexusRaven accepts a list of python functions. These python functions can do anything (including sending GET/POST requests to external APIs!). The two requirements include the python function signature and the appropriate docstring to generate the function call. 
-
-NexusRaven is highly compatible with langchain. See [langchain_example.py](https://huggingface.co/Nexusflow/NexusRaven-13B/blob/main/langchain_example.py). An example without langchain can be found in [non_langchain_example.py](https://huggingface.co/Nexusflow/NexusRaven-13B/blob/main/non_langchain_example.py).
-
-Please note that the model will reflect on the answer sometimes, so we highly recommend stopping the model generation at a stopping criteria of `["\nReflection:"]`, to avoid spending unnecessary tokens during inference, but the reflection might help in some rare cases. This is reflected in our langchain example. 
-
-More information about how to prompt the model can be found in [prompting_readme.md](prompting_readme.md). 
-
-The "Initial Answer" can be executed to run the function.
-
-### Quickstart
-You can run the model on a GPU using the following code. 
-```python
-# Please `pip install transformers accelerate`
-from transformers import pipeline
-
-
-pipeline = pipeline(
-    "text-generation",
-    model="Nexusflow/NexusRaven-13B",
-    torch_dtype="auto",
-    device_map="auto",
-)
-
-prompt_template = """
-<human>:
-OPTION:
-<func_start>def hello_world(n : int)<func_end>
-<docstring_start>
-\"\"\"
-Prints hello world to the user.
-
-Args:
-n (int) : Number of times to print hello world.
-\"\"\"
-<docstring_end>
-
-OPTION:
-<func_start>def hello_universe(n : int)<func_end>
-<docstring_start>
-\"\"\"
-Prints hello universe to the user.
-
-Args:
-n (int) : Number of times to print hello universe.
-\"\"\"
-<docstring_end>
-
-User Query: Question: {question}
-
-Please pick a function from the above options that best answers the user query and fill in the appropriate arguments.<human_end>
-"""
-prompt = prompt_template.format(question="Please print hello world 10 times.")
-
-result = pipeline(prompt, max_new_tokens=100, return_full_text=False, do_sample=False)[0]["generated_text"]
-
-# Get the "Initial Call" only
-start_str = "Initial Answer: "
-end_str = "\nReflection: "
-start_idx = result.find(start_str) + len(start_str)
-end_idx = result.find(end_str)
-function_call = result[start_idx: end_idx]
-
-print (f"Generated Call: {function_call}")
+E.g.
 ```
-This will output: 
-```text
-Generated Call: hello_world(10) 
+text = "<s>[INST] What is your favourite condiment? [/INST]"
+"Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> "
+"[INST] Do you have mayonnaise recipes? [/INST]"
 ```
-Which can be executed. 
-
 
-## Training procedure
+This format is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating) via the `apply_chat_template()` method:
 
-### Training hyperparameters
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
-The following hyperparameters were used during training:
-- learning_rate: 3e-05
-- train_batch_size: 1
-- eval_batch_size: 1
-- seed: 42
-- distributed_type: multi-GPU
-- num_devices: 8
-- gradient_accumulation_steps: 16
-- total_train_batch_size: 128
-- total_eval_batch_size: 8
-- optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-08
-- lr_scheduler_type: constant
-- num_epochs: 2.0
+device = "cuda" # the device to load the model onto
 
+model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
+tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
 
-### Framework versions
+messages = [
+    {"role": "user", "content": "What is your favourite condiment?"},
+    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
+    {"role": "user", "content": "Do you have mayonnaise recipes?"}
+]
 
-- Transformers 4.33.2
-- Pytorch 2.0.1+cu118
-- Datasets 2.14.5
-- Tokenizers 0.13.3
+encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
 
+model_inputs = encodeds.to(device)
+model.to(device)
 
-# Limitations
-1. We highly recommend using a stop criteria of `["\nReflection:"]`. The model was trained to first generate an answer and then reflect on its answer to either improve the answer or keep the answer the same. However, this "chain of thought" is often not helpful, and the final answer is seldom better than the initial call. Therefore, we strongly recommend using the Initial Call as the main call to execute. 
-2. The model works best when it is connected with a retriever when there are a multitude of functions, as a large number of functions will saturate the context window of this model. 
-3. The model can be prone to generate incorrect calls. Please ensure proper guardrails to capture errant behavior is in place. 
+generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
+decoded = tokenizer.batch_decode(generated_ids)
+print(decoded[0])
+```
 
+## Model Architecture
+This instruction model is based on Mistral-7B-v0.1, a transformer model with the following architecture choices:
+- Grouped-Query Attention
+- Sliding-Window Attention
+- Byte-fallback BPE tokenizer
 
-## License
-This model was trained on commercially viable data and is licensed under the [Llama 2 community license](https://huggingface.co/codellama/CodeLlama-13b-hf/blob/main/LICENSE) following the original [CodeLlama-13b-hf](https://huggingface.co/codellama/CodeLlama-13b-hf/) model. 
+## Troubleshooting
+- If you see the following error:
+```
+Traceback (most recent call last):
+File "", line 1, in
+File "/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
+config, kwargs = AutoConfig.from_pretrained(
+File "/transformers/models/auto/configuration_auto.py", line 1022, in from_pretrained
+config_class = CONFIG_MAPPING[config_dict["model_type"]]
+File "/transformers/models/auto/configuration_auto.py", line 723, in getitem
+raise KeyError(key)
+KeyError: 'mistral'
+```
 
+Installing transformers from source should solve the issue
+pip install git+https://github.com/huggingface/transformers
 
-## References
-We thank the CodeLlama team for their amazing models!
+This should not be required after transformers-v4.33.4.
 
-```
-@misc{rozière2023code,
-      title={Code Llama: Open Foundation Models for Code}, 
-      author={Baptiste Rozière and Jonas Gehring and Fabian Gloeckle and Sten Sootla and Itai Gat and Xiaoqing Ellen Tan and Yossi Adi and Jingyu Liu and Tal Remez and Jérémy Rapin and Artyom Kozhevnikov and Ivan Evtimov and Joanna Bitton and Manish Bhatt and Cristian Canton Ferrer and Aaron Grattafiori and Wenhan Xiong and Alexandre Défossez and Jade Copet and Faisal Azhar and Hugo Touvron and Louis Martin and Nicolas Usunier and Thomas Scialom and Gabriel Synnaeve},
-      year={2023},
-      eprint={2308.12950},
-      archivePrefix={arXiv},
-      primaryClass={cs.CL}
-}
-```
+## Limitations
 
+The Mistral 7B Instruct model is a quick demonstration that the base model can be easily fine-tuned to achieve compelling performance. 
+It does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to
+make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.
 
-## Citation
-```
-@misc{nexusraven,
-      title={NexusRaven: Surpassing the state-of-the-art in open-source function calling LLMs}, 
-      author={Nexusflow.ai team},
-      year={2023},
-      url={http://nexusflow.ai/blog}
-}
-```
+## The Mistral AI Team
 
-## Contact
-Please reach out to [email protected] for any questions!
+Albert Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed.