Update README.md
Browse files
README.md
CHANGED
@@ -17,16 +17,14 @@ base_model:
|
|
17 |
pipeline_tag: text-generation
|
18 |
---
|
19 |
|
20 |
-
#
|
|
|
|
|
21 |
```
|
22 |
pip install git+https://github.com/huggingface/transformers@main
|
23 |
pip install --pre torchao --index-url https://download.pytorch.org/whl/nightly/cu126
|
24 |
-
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
25 |
```
|
26 |
|
27 |
-
Also need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation-harness#install
|
28 |
-
|
29 |
-
# Quantization Recipe
|
30 |
We used following code to get the quantized model:
|
31 |
|
32 |
```
|
@@ -119,6 +117,8 @@ Hello! As an AI, I don't have consciousness in the way humans do, but I am fully
|
|
119 |
|
120 |
We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
|
121 |
|
|
|
|
|
122 |
## baseline
|
123 |
```
|
124 |
lm_eval --model hf --model_args pretrained=microsoft/Phi-4-mini-instruct --tasks hellaswag --device cuda:0 --batch_size 64
|
|
|
17 |
pipeline_tag: text-generation
|
18 |
---
|
19 |
|
20 |
+
# Quantization Recipe
|
21 |
+
|
22 |
+
First need to install the required packages:
|
23 |
```
|
24 |
pip install git+https://github.com/huggingface/transformers@main
|
25 |
pip install --pre torchao --index-url https://download.pytorch.org/whl/nightly/cu126
|
|
|
26 |
```
|
27 |
|
|
|
|
|
|
|
28 |
We used following code to get the quantized model:
|
29 |
|
30 |
```
|
|
|
117 |
|
118 |
We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
|
119 |
|
120 |
+
Need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation-harness#install
|
121 |
+
|
122 |
## baseline
|
123 |
```
|
124 |
lm_eval --model hf --model_args pretrained=microsoft/Phi-4-mini-instruct --tasks hellaswag --device cuda:0 --batch_size 64
|