jinjieyuan
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -3,7 +3,7 @@ language: en
|
|
3 |
license: apache-2.0
|
4 |
---
|
5 |
|
6 |
-
# Shears Model Card: shears-mpt-7b-50-gsm8k-super
|
7 |
|
8 |
The super-adapter fine-tuned on sparsified [MPT-7B](https://huggingface.co/mosaicml/mpt-7b) with GSM8K datasets using Shears.
|
9 |
|
@@ -13,11 +13,11 @@ The release of the super-network is to facilitate users to apply their own searc
|
|
13 |
|
14 |
### Information
|
15 |
|
16 |
-
- **Model name:** shears-mpt-7b-50-gsm8k-super
|
17 |
- **Base model:** [IntelLabs/shears-mpt-7b-50-base](https://huggingface.co/IntelLabs/shears-mpt-7b-50-base)
|
18 |
- **Sparsity:** 50%
|
19 |
- **Subnetwork version:** Super
|
20 |
-
- **NNCF Configuration:** [nncf_shears_mpt.json](https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/
|
21 |
|
22 |
### Adapter Configuration
|
23 |
|
@@ -59,7 +59,7 @@ def generate_prompt(instruction):
|
|
59 |
"""
|
60 |
|
61 |
base_model = AutoModelForCausalLM.from_pretrained("IntelLabs/shears-mpt-7b-50-base", trust_remote_code=True)
|
62 |
-
model = PeftModel.from_pretrained(base_model, "IntelLabs/shears-mpt-7b-50-gsm8k-super")
|
63 |
model.eval()
|
64 |
|
65 |
non_zero_params = sum([(param.data != 0).sum().item() for _, param in model.named_parameters()])
|
|
|
3 |
license: apache-2.0
|
4 |
---
|
5 |
|
6 |
+
# Shears Model Card: shears-mpt-7b-50-gsm8k-super-adapter
|
7 |
|
8 |
The super-adapter fine-tuned on sparsified [MPT-7B](https://huggingface.co/mosaicml/mpt-7b) with GSM8K datasets using Shears.
|
9 |
|
|
|
13 |
|
14 |
### Information
|
15 |
|
16 |
+
- **Model name:** shears-mpt-7b-50-gsm8k-super-adapter
|
17 |
- **Base model:** [IntelLabs/shears-mpt-7b-50-base](https://huggingface.co/IntelLabs/shears-mpt-7b-50-base)
|
18 |
- **Sparsity:** 50%
|
19 |
- **Subnetwork version:** Super
|
20 |
+
- **NNCF Configuration:** [nncf_shears_mpt.json](https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/blob/main/Shears/nncf_config/nncf_shears_mpt.json)
|
21 |
|
22 |
### Adapter Configuration
|
23 |
|
|
|
59 |
"""
|
60 |
|
61 |
base_model = AutoModelForCausalLM.from_pretrained("IntelLabs/shears-mpt-7b-50-base", trust_remote_code=True)
|
62 |
+
model = PeftModel.from_pretrained(base_model, "IntelLabs/shears-mpt-7b-50-gsm8k-super-adapter")
|
63 |
model.eval()
|
64 |
|
65 |
non_zero_params = sum([(param.data != 0).sum().item() for _, param in model.named_parameters()])
|