jyhong836 commited on
Commit
80bad79
·
1 Parent(s): 61199cd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -18,6 +18,20 @@ The models are prepared by [Visual Informatics Group @ University of Texas at Au
18
 
19
  License: [MIT License](https://opensource.org/license/mit/)
20
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
  | | Base Model | Model Size | Compression Method | Compression Degree |
23
  |---:|:-------------|:-------------|:-----------------------|:--------------------------------------------------------------------------------------|
 
18
 
19
  License: [MIT License](https://opensource.org/license/mit/)
20
 
21
+ How to use
22
+ ```python
23
+ base_model = 'llama-2-7b'
24
+ comp_degree = 0.1
25
+ comp_method = 'sparsegpt_unstructured'
26
+ model_path = f'vita-group/comp-{arch}_{comp_method}_s{comp_degree}'
27
+ model = AutoModelForCausalLM.from_pretrained(
28
+ model_path,
29
+ torch_dtype=torch.float16,
30
+ low_cpu_mem_usage=True,
31
+ device_map="auto"
32
+ )
33
+ ```
34
+
35
 
36
  | | Base Model | Model Size | Compression Method | Compression Degree |
37
  |---:|:-------------|:-------------|:-----------------------|:--------------------------------------------------------------------------------------|