|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- Open-Orca/SlimOrca |
|
--- |
|
## Usage |
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("serpdotai/sparsetral-16x7B-v1", trust_remote_code=True) |
|
model = AutoModelForCausalLM.from_pretrained("serpdotai/sparsetral-16x7B-v1", device_map="auto", trust_remote_code=True).eval() |
|
|
|
inputs = tokenizer('### Human:\nHow are you?\n### Assistant:\n', return_tensors='pt') |
|
inputs = inputs.to(model.device) |
|
pred = model.generate(**inputs) |
|
print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True)) |
|
# I am doing well, thank you. |
|
``` |
