Updated the RAM usage chart for the ggmlv3 models

README.md

@@ -12,11 +12,34 @@ datasets:
 
 ### This repository contains quantized conversions of EleutherAI's Pythia Deduped checkpoints.
 
-Notes:
-- Converted with ggerganov/ggml's gpt-neox conversion script, and tested with KoboldCpp.
-- I can't promise that this will work, especially with other frontends. ~~I've had problems when generating words like "Alice" or "Hakurei" / "Gensokyo". Could be related to the ggml implementation of GPT-NeoX having a "hacked" tokenizer [(source)](https://github.com/ggerganov/ggml/tree/master/examples/gpt-neox#notes).~~ **This seems to have been improved with KoboldCpp v1.25.1 and the ggmlv3 versions of these models.**
+If you're starting off, I highly recommend for you to get models from the newest directory [(2023-05-25)](https://huggingface.co/Merry/ggml-pythia-deduped/tree/main/2023-05-25).
 
-Versions:
+# RAM USAGE
+Model | RAM usage
+:--:|:--:
+Unloaded | 41.3 MiB
+|
+ggmlv3-pythia-70m-deduped-q4_0.bin | 95.5 MiB
+ggmlv3-pythia-160m-deduped-q4_0.bin | 201.1 MiB
+ggmlv3-pythia-410m-deduped-q4_0.bin | 415.1 MiB
+ggmlv3-pythia-1b-deduped-q4_0.bin | 762.2 MiB
+ggmlv3-pythia-1.4b-deduped-q4_0.bin | 1.0 GiB
+ggmlv3-pythia-2.8b-deduped-q4_0.bin | 1.9 GiB
+|
+ggmlv3-pythia-70m-deduped-q5_1.bin | 108.7 MiB
+ggmlv3-pythia-160m-deduped-q5_1.bin | 226.9 MiB
+ggmlv3-pythia-410m-deduped-q5_1.bin | 494.0 MiB
+ggmlv3-pythia-1b-deduped-q5_1.bin | 943.9 MiB
+ggmlv3-pythia-1.4b-deduped-q5_1.bin | 1.3 GiB
+ggmlv3-pythia-2.8b-deduped-q5_1.bin | 2.3 GiB
+
+*Tested on KoboldCpp with OpenBLAS enabled.*
+
+**Notes:**
+- The models have been converted with ggerganov/ggml's gpt-neox conversion script, and tested only on KoboldCpp. Other frontends that support GGML-based conversions of GPT-NeoX *should* work, but I can't promise anything.
+- They're sorted by date based on when they were made so it was easier to track breaking changes. If you're just starting off I highly recommend the latest (which is 2023-05-25). Combined with KoboldCpp v1.25.1+ this improved the tokenizer, which in my testing reduces occurrences of broken words like "Alicae" or "Reimu Hai-ku-rei".
+
+**Versions:**
 
 **2023-04-20:** *q4_3. Used [commit 05f3079](https://github.com/ggerganov/ggml/tree/05f307971862b83df12fada0c42ee027ba5a82b5/examples/stablelm)*
 
@@ -26,27 +49,10 @@ Versions:
 
 **2023-05-15:** *New quantization format (ggmlv2). q4_0 and q5_1, up to 2.8B. Used [commit 010203f](https://github.com/ggerganov/ggml/tree/010203f94a85df5c86b773dc5acb698c8e7b1e7b/examples/gpt-neox)*
 
-**2023-05-25 (RECOMMENDED):** *New quantization format (ggmlv3). q4_0 and q5_1, up to 2.8B. Used [commit 73ad593](https://github.com/ggerganov/ggml/tree/73ad593cf84f864f0fcfd3a196253575c70d66a2/examples/gpt-neox)*
+**2023-05-25:** *New quantization format (ggmlv3). q4_0 and q5_1, up to 2.8B. Used [commit 73ad593](https://github.com/ggerganov/ggml/tree/73ad593cf84f864f0fcfd3a196253575c70d66a2/examples/gpt-neox)*
 
 They're separated by date and commit so it's easier to track of any breaking changes.
 
-# RAM USAGE (on KoboldCpp w/ OpenBLAS)
-Model | Initial RAM | After generation
-:--:|:--:|:--:
-Unloaded | 41.3 MiB
-ggml-pythia-70m-deduped-q4_0.bin | 113.3 MiB | 267.8 MiB
-ggml-pythia-70m-deduped-q5_1.bin | 121.5 MiB | 129.4 MiB
-ggml-pythia-160m-deduped-q4_0.bin | 199.4 MiB | 201.6 MiB
-ggml-pythia-160m-deduped-q5_1.bin | 227.5 MiB | 241.0 MiB
-ggml-pythia-410m-deduped-q4_0.bin | 399.2 MiB | 406.2 MiB
-ggml-pythia-410m-deduped-q5_1.bin | 455.7 MiB | 460.3 MiB
-ggml-pythia-1b-deduped-q4_0.bin | 803.0 MiB | 809.0 MiB
-ggml-pythia-1b-deduped-q5_1.bin | 921.5 MiB | 927.3 MiB
-ggml-pythia-1.4b-deduped-q4_0.bin | 1.1 GiB | 1.1 GiB
-ggml-pythia-1.4b-deduped-q5_1.bin | 1.3 GiB | 1.3 GiB
-ggml-pythia-2.8b-deduped-q4_0.bin | 2.0 GiB | 2.0 GiB
-ggml-pythia-2.8b-deduped-q5_1.bin | 2.4 GiB | 2.4 GiB
-
 # ALTERNATIVES
 If you're here because you want a smaller model to run on a device with constrained memory, consider the following:
 - OpenLLaMA [3B](https://huggingface.co/openlm-research/open_llama_3b_350bt_preview) [(7B)](https://huggingface.co/openlm-research/open_llama_7b_400bt_preview)