jerryzh168 commited on
Commit
53e8363
·
verified ·
1 Parent(s): eef2ace

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -17,16 +17,14 @@ base_model:
17
  pipeline_tag: text-generation
18
  ---
19
 
20
- # Installation
 
 
21
  ```
22
  pip install git+https://github.com/huggingface/transformers@main
23
  pip install --pre torchao --index-url https://download.pytorch.org/whl/nightly/cu126
24
- pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
25
  ```
26
 
27
- Also need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation-harness#install
28
-
29
- # Quantization Recipe
30
  We used following code to get the quantized model:
31
 
32
  ```
@@ -119,6 +117,8 @@ Hello! As an AI, I don't have consciousness in the way humans do, but I am fully
119
 
120
  We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
121
 
 
 
122
  ## baseline
123
  ```
124
  lm_eval --model hf --model_args pretrained=microsoft/Phi-4-mini-instruct --tasks hellaswag --device cuda:0 --batch_size 64
 
17
  pipeline_tag: text-generation
18
  ---
19
 
20
+ # Quantization Recipe
21
+
22
+ First need to install the required packages:
23
  ```
24
  pip install git+https://github.com/huggingface/transformers@main
25
  pip install --pre torchao --index-url https://download.pytorch.org/whl/nightly/cu126
 
26
  ```
27
 
 
 
 
28
  We used following code to get the quantized model:
29
 
30
  ```
 
117
 
118
  We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
119
 
120
+ Need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation-harness#install
121
+
122
  ## baseline
123
  ```
124
  lm_eval --model hf --model_args pretrained=microsoft/Phi-4-mini-instruct --tasks hellaswag --device cuda:0 --batch_size 64