Hugging Face's logo

Spaces:
vita-group
/
README
Running

App Files Community
jyhong836 commited on
Commit
56954b0
·
1 Parent(s): 61850b4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +23 -23
README.md CHANGED
@@ -33,37 +33,37 @@ How to use
33
  ```python
34
  from transformers import AutoModelForCausalLM, AutoTokenizer
35
  base_model = 'llama-2-7b'
36
- comp_degree = 0.1
37
- comp_method = 'sparsegpt_unstructured'
38
- model_path = f'vita-group/comp-{arch}_{comp_method}_s{comp_degree}'
39
  model = AutoModelForCausalLM.from_pretrained(
40
  model_path,
 
41
  torch_dtype=torch.float16,
42
  low_cpu_mem_usage=True,
43
  device_map="auto"
44
  )
45
- tokenizer = AutoTokenizer.from_pretrained('meta-llama/Llama-2-7b')
46
  input_ids = tokenizer('Hello! I am a VITA-compressed-LLM chatbot!', return_tensors='pt').input_ids
47
  outputs = model.generate(input_ids)
 
48
  ```
49
 
50
 
51
- | | Base Model | Model Size | Compression Method | Compression Degree |
52
- |---:|:-------------|:-------------|:-----------------------|:--------------------------------------------------------------------------------------|
53
- | 0 | Llama-2 | 7b | magnitude_unstructured | [s0.1](https://huggingface.co/vita-group/comp-llama-2-7b_magnitude_unstructured_s0.1) |
54
- | 1 | Llama-2 | 7b | magnitude_unstructured | [s0.2](https://huggingface.co/vita-group/comp-llama-2-7b_magnitude_unstructured_s0.2) |
55
- | 2 | Llama-2 | 7b | magnitude_unstructured | [s0.3](https://huggingface.co/vita-group/comp-llama-2-7b_magnitude_unstructured_s0.3) |
56
- | 3 | Llama-2 | 7b | magnitude_unstructured | [s0.5](https://huggingface.co/vita-group/comp-llama-2-7b_magnitude_unstructured_s0.5) |
57
- | 4 | Llama-2 | 7b | magnitude_unstructured | [s0.6](https://huggingface.co/vita-group/comp-llama-2-7b_magnitude_unstructured_s0.6) |
58
- | 5 | Llama-2 | 7b | sparsegpt_unstructured | [s0.1](https://huggingface.co/vita-group/comp-llama-2-7b_sparsegpt_unstructured_s0.1) |
59
- | 6 | Llama-2 | 7b | sparsegpt_unstructured | [s0.2](https://huggingface.co/vita-group/comp-llama-2-7b_sparsegpt_unstructured_s0.2) |
60
- | 7 | Llama-2 | 7b | sparsegpt_unstructured | [s0.3](https://huggingface.co/vita-group/comp-llama-2-7b_sparsegpt_unstructured_s0.3) |
61
- | 8 | Llama-2 | 7b | sparsegpt_unstructured | [s0.5](https://huggingface.co/vita-group/comp-llama-2-7b_sparsegpt_unstructured_s0.5) |
62
- | 9 | Llama-2 | 7b | sparsegpt_unstructured | [s0.6](https://huggingface.co/vita-group/comp-llama-2-7b_sparsegpt_unstructured_s0.6) |
63
- | 10 | Llama-2 | 7b | wanda_unstructured | [s0.1](https://huggingface.co/vita-group/comp-llama-2-7b_wanda_unstructured_s0.1) |
64
- | 11 | Llama-2 | 7b | wanda_unstructured | [s0.2](https://huggingface.co/vita-group/comp-llama-2-7b_wanda_unstructured_s0.2) |
65
- | 12 | Llama-2 | 7b | wanda_unstructured | [s0.3](https://huggingface.co/vita-group/comp-llama-2-7b_wanda_unstructured_s0.3) |
66
- | 13 | Llama-2 | 7b | wanda_unstructured | [s0.5](https://huggingface.co/vita-group/comp-llama-2-7b_wanda_unstructured_s0.5) |
67
- | 14 | Llama-2 | 7b | wanda_unstructured | [s0.6](https://huggingface.co/vita-group/comp-llama-2-7b_wanda_unstructured_s0.6) |
68
-
69
-
 
33
  ```python
34
  from transformers import AutoModelForCausalLM, AutoTokenizer
35
  base_model = 'llama-2-7b'
36
+ comp_method = 'magnitude_unstructured'
37
+ comp_degree = 0.2
38
+ model_path = f'vita-group/{base_model}_{comp_method}'
39
  model = AutoModelForCausalLM.from_pretrained(
40
  model_path,
41
+ revision=f's{comp_degree}',
42
  torch_dtype=torch.float16,
43
  low_cpu_mem_usage=True,
44
  device_map="auto"
45
  )
46
+ tokenizer = AutoTokenizer.from_pretrained('meta-llama/Llama-2-7b-hf')
47
  input_ids = tokenizer('Hello! I am a VITA-compressed-LLM chatbot!', return_tensors='pt').input_ids
48
  outputs = model.generate(input_ids)
49
+ print(tokenizer.decode(outputs[0]))
50
  ```
51
 
52
 
53
+ | | Base Model | Model Size | Compression Method | Compression Degree |
54
+ |---:|:-------------|:-------------|:----------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------|
55
+ | 0 | Llama-2 | 7b | [magnitude_unstructured](https://huggingface.co/vita-group/llama-2-7b_magnitude_unstructured) | [s0.1](https://huggingface.co/vita-group/llama-2-7b_magnitude_unstructured/tree/s0.1) |
56
+ | 1 | Llama-2 | 7b | [magnitude_unstructured](https://huggingface.co/vita-group/llama-2-7b_magnitude_unstructured) | [s0.2](https://huggingface.co/vita-group/llama-2-7b_magnitude_unstructured/tree/s0.2) |
57
+ | 2 | Llama-2 | 7b | [magnitude_unstructured](https://huggingface.co/vita-group/llama-2-7b_magnitude_unstructured) | [s0.3](https://huggingface.co/vita-group/llama-2-7b_magnitude_unstructured/tree/s0.3) |
58
+ | 3 | Llama-2 | 7b | [magnitude_unstructured](https://huggingface.co/vita-group/llama-2-7b_magnitude_unstructured) | [s0.5](https://huggingface.co/vita-group/llama-2-7b_magnitude_unstructured/tree/s0.5) |
59
+ | 4 | Llama-2 | 7b | [magnitude_unstructured](https://huggingface.co/vita-group/llama-2-7b_magnitude_unstructured) | [s0.6](https://huggingface.co/vita-group/llama-2-7b_magnitude_unstructured/tree/s0.6) |
60
+ | 5 | Llama-2 | 7b | [sparsegpt_unstructured](https://huggingface.co/vita-group/llama-2-7b_sparsegpt_unstructured) | [s0.1](https://huggingface.co/vita-group/llama-2-7b_sparsegpt_unstructured/tree/s0.1) |
61
+ | 6 | Llama-2 | 7b | [sparsegpt_unstructured](https://huggingface.co/vita-group/llama-2-7b_sparsegpt_unstructured) | [s0.2](https://huggingface.co/vita-group/llama-2-7b_sparsegpt_unstructured/tree/s0.2) |
62
+ | 7 | Llama-2 | 7b | [sparsegpt_unstructured](https://huggingface.co/vita-group/llama-2-7b_sparsegpt_unstructured) | [s0.3](https://huggingface.co/vita-group/llama-2-7b_sparsegpt_unstructured/tree/s0.3) |
63
+ | 8 | Llama-2 | 7b | [sparsegpt_unstructured](https://huggingface.co/vita-group/llama-2-7b_sparsegpt_unstructured) | [s0.5](https://huggingface.co/vita-group/llama-2-7b_sparsegpt_unstructured/tree/s0.5) |
64
+ | 9 | Llama-2 | 7b | [sparsegpt_unstructured](https://huggingface.co/vita-group/llama-2-7b_sparsegpt_unstructured) | [s0.6](https://huggingface.co/vita-group/llama-2-7b_sparsegpt_unstructured/tree/s0.6) |
65
+ | 10 | Llama-2 | 7b | [wanda_unstructured](https://huggingface.co/vita-group/llama-2-7b_wanda_unstructured) | [s0.1](https://huggingface.co/vita-group/llama-2-7b_wanda_unstructured/tree/s0.1) |
66
+ | 11 | Llama-2 | 7b | [wanda_unstructured](https://huggingface.co/vita-group/llama-2-7b_wanda_unstructured) | [s0.2](https://huggingface.co/vita-group/llama-2-7b_wanda_unstructured/tree/s0.2) |
67
+ | 12 | Llama-2 | 7b | [wanda_unstructured](https://huggingface.co/vita-group/llama-2-7b_wanda_unstructured) | [s0.3](https://huggingface.co/vita-group/llama-2-7b_wanda_unstructured/tree/s0.3) |
68
+ | 13 | Llama-2 | 7b | [wanda_unstructured](https://huggingface.co/vita-group/llama-2-7b_wanda_unstructured) | [s0.5](https://huggingface.co/vita-group/llama-2-7b_wanda_unstructured/tree/s0.5) |
69
+ | 14 | Llama-2 | 7b | [wanda_unstructured](https://huggingface.co/vita-group/llama-2-7b_wanda_unstructured) | [s0.6](https://huggingface.co/vita-group/llama-2-7b_wanda_unstructured/tree/s0.6) |