File size: 2,116 Bytes
43928c8 31e311e 0b60abf 09eb6a9 58780e5 09eb6a9 58780e5 09eb6a9 58780e5 09eb6a9 5a92d8d 43928c8 31e311e 60e56e1 9569d91 31e311e 05e7d8f 31e311e 0b60abf |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
---
license: cc-by-nc-sa-4.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- stableLM
- sharded
widget:
- text: Imagine Einstein was part of a comedy duo. What would be their stage name?
example_title: Einstein's comedy duo
- text: What do you think Einstein's favorite Swiss chocolate brand would be?
example_title: Einstein's chocolate
- text: If Einstein were to enter a yodeling competition in Switzerland, what
would his yodel sound like?
example_title: Einstein's yodel
- text: If Einstein had to create a Swiss-themed superhero, what would their name
and superpower be?
example_title: Swiss superhero
- text: What kind of wild party would Einstein throw at ETH Zurich?
example_title: Einstein's party
- text: If Einstein had a pet Swiss cow, what would he name it and why?
example_title: Einstein's cow
- text: You've discovered a secret Swiss cheese that grants the power of genius.
How would you use it to become the next Einstein?
example_title: Genius cheese
inference:
parameters:
max_length: 96
min_length: 32
---
# StableLM-Tuned-Alpha: sharded checkpoint
<a href="https://colab.research.google.com/gist/pszemraj/4bd75aa3744f2a02a5c0ee499932b7eb/sharded-stablelm-testing-notebook.ipynb">
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
</a>
This is a sharded checkpoint (with ~2GB shards) of the model. Refer to the [original model](https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b) for all details.
## Basic Usage
install `transformers`, `accelerate`, and `bitsandbytes`.
```bash
pip install -U -q transformers bitsandbytes accelerate
```
Load the model in 8bit, then [run inference](https://huggingface.co/docs/transformers/generation_strategies#contrastive-search):
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "ethzanalytics/stablelm-tuned-alpha-3b-sharded"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, load_in_8bit=True)
``` |