Benchmarks added
Browse files
README.md
CHANGED
|
@@ -38,6 +38,18 @@ Refs:
|
|
| 38 |
<br>
|
| 39 |
<br>
|
| 40 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
# Basic installation procedure
|
| 42 |
|
| 43 |
- It was a nightmare, I will only detail briefly what you'll need. WSL was quite painful to sort out.
|
|
@@ -82,7 +94,7 @@ pip install -r requirements.txt
|
|
| 82 |
<br>
|
| 83 |
<br>
|
| 84 |
|
| 85 |
-
# Testbench detail and
|
| 86 |
|
| 87 |
- Latest version of oobabooga + https://github.com/oobabooga/text-generation-webui/pull/1229
|
| 88 |
|
|
|
|
| 38 |
<br>
|
| 39 |
<br>
|
| 40 |
|
| 41 |
+
# Benchmarks
|
| 42 |
+
|
| 43 |
+
Best results in **bold**.
|
| 44 |
+
|
| 45 |
+
| Model | wikitext2 PPL | c4 PPL | ptb PPL | C4-VRAM |
|
| 46 |
+
|---|---|---|---|---|---|
|
| 47 |
+
| 4bit-GPTQ - TheBloke/vicuna-13B-1.1-GPTQ-4bit-128g | 8.517391204833984 | **7.058407783508301** | 20.888103485107422 | **8670.26953125** |
|
| 48 |
+
| 8bit-GPTQ - Thireus/Vicuna13B-v1.1-8bit-128g | **8.508771896362305** | 7.105874538421631 | **20.75649070739746** | 14840.26171875 |
|
| 49 |
+
|
| 50 |
+
<br>
|
| 51 |
+
<br>
|
| 52 |
+
|
| 53 |
# Basic installation procedure
|
| 54 |
|
| 55 |
- It was a nightmare, I will only detail briefly what you'll need. WSL was quite painful to sort out.
|
|
|
|
| 94 |
<br>
|
| 95 |
<br>
|
| 96 |
|
| 97 |
+
# Testbench detail and demo
|
| 98 |
|
| 99 |
- Latest version of oobabooga + https://github.com/oobabooga/text-generation-webui/pull/1229
|
| 100 |
|