File size: 1,028 Bytes
c9d1fec f2b6d28 c9d1fec |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
---
library_name: peft
base_model: meta-llama/Llama-2-7b-chat-hf
---
# Model Card for nicce/sexbot
This is a lora adapter for mistralai/Mixtral-8x7b-Instruct-v0.1
## Testing the Model
To quickly test the model, you can run it on a GPU with the transformers / peft library:
```python
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("nicce/sexbot")
model = AutoPeftModelForCausalLM.from_pretrained("nicce/sexbot").to("cuda") # if you get a CUDA out of memory error, try load_in_8bit=True
messages = [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "Hi, can you please explain machine learning to me?"}
]
encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
generated_ids = model.generate(input_ids=model_inputs, min_new_tokens=10, max_new_tokens=300, do_sample=True, temperature=0.9, top_p=0.8)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])
``` |