marcodambra commited on
Commit
99c461a
·
verified ·
1 Parent(s): 3db21aa

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -46,7 +46,7 @@ llm = Llama(
46
  model_path="/path/to/model.gguf", # Download the model file first
47
  n_ctx=2048, # The max sequence length to use - note that longer sequence lengths require much more resources
48
  n_threads=8, # The number of CPU threads to use, tailor to your system and the resulting performance
49
- n_gpu_layers=35 # The number of layers to offload to GPU, if you have GPU acceleration available
50
  )
51
 
52
  # Simple inference example
@@ -79,7 +79,7 @@ print(assistant_message)
79
 
80
  ## Bias, Risks and Limitations
81
 
82
- Azzurro-Quantized and its original model [Azzurro](https://huggingface.co/MoxoffSpA/Azzurro) have not been aligned to human preferences for safety within the RLHF phase or deployed with in-the-loop filtering of
83
  responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition
84
  of the corpus were used to train the base model [mistralai/Mistral-7B-v0.2](https://huggingface.co/mistralai/Mistral-7B-v0.2), however, it is likely to have included a mix of Web data and technical sources
85
  like books and code.
 
46
  model_path="/path/to/model.gguf", # Download the model file first
47
  n_ctx=2048, # The max sequence length to use - note that longer sequence lengths require much more resources
48
  n_threads=8, # The number of CPU threads to use, tailor to your system and the resulting performance
49
+ n_gpu_layers=0 # The number of layers to offload to GPU, if you have GPU acceleration available
50
  )
51
 
52
  # Simple inference example
 
79
 
80
  ## Bias, Risks and Limitations
81
 
82
+ AzzurroQuantized and its original model [Azzurro](https://huggingface.co/MoxoffSpA/Azzurro) have not been aligned to human preferences for safety within the RLHF phase or deployed with in-the-loop filtering of
83
  responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition
84
  of the corpus were used to train the base model [mistralai/Mistral-7B-v0.2](https://huggingface.co/mistralai/Mistral-7B-v0.2), however, it is likely to have included a mix of Web data and technical sources
85
  like books and code.