Hugging Face's logo

nm-testing
/
llava-1.5-7b-hf-FP8-dynamic

Safetensors
llava
Model card Files Community
1
mgoin commited on
Commit
1ea70b6
·
verified ·
1 Parent(s): c6cc8f2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +29 -0
README.md CHANGED
@@ -1,3 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  https://github.com/vllm-project/llm-compressor/pull/185
2
 
3
  ```python
 
1
+
2
+ ## Evaluation
3
+
4
+ ```
5
+ lm_eval --model vllm-vlm --model_args pretrained=llava-hf/llava-1.5-7b-hf --tasks mmmu_val
6
+ | Groups |Version|Filter|n-shot|Metric| |Value | |Stderr|
7
+ |--------------------------------|------:|------|------|------|---|-----:|---|-----:|
8
+ |mmmu_val | 0|none | |acc |↑ |0.2433|± |0.0141|
9
+ | - Art and Design | 0|none | |acc |↑ |0.2250|± |0.0384|
10
+ | - Business | 0|none | |acc |↑ |0.2600|± |0.0358|
11
+ | - Health and Medicine | 0|none | |acc |↑ |0.3067|± |0.0377|
12
+ | - Humanities and Social Science| 0|none | |acc |↑ |0.2667|± |0.0403|
13
+ | - Science | 0|none | |acc |↑ |0.1667|± |0.0308|
14
+ | - Tech and Engineering | 0|none | |acc |↑ |0.2381|± |0.0284|
15
+
16
+
17
+ lm_eval --model vllm-vlm --model_args pretrained=mgoin/llava-1.5-7b-hf-FP8-Dynamic --tasks mmmu_val
18
+ | Groups |Version|Filter|n-shot|Metric| |Value | |Stderr|
19
+ |--------------------------------|------:|------|------|------|---|-----:|---|-----:|
20
+ |mmmu_val | 0|none | |acc |↑ |0.2433|± |0.0141|
21
+ | - Art and Design | 0|none | |acc |↑ |0.2250|± |0.0384|
22
+ | - Business | 0|none | |acc |↑ |0.2600|± |0.0358|
23
+ | - Health and Medicine | 0|none | |acc |↑ |0.3067|± |0.0377|
24
+ | - Humanities and Social Science| 0|none | |acc |↑ |0.2667|± |0.0403|
25
+ | - Science | 0|none | |acc |↑ |0.1667|± |0.0308|
26
+ | - Tech and Engineering | 0|none | |acc |↑ |0.2381|± |0.0284|
27
+ ```
28
+
29
+ ## Creation
30
  https://github.com/vllm-project/llm-compressor/pull/185
31
 
32
  ```python