Update README.md

README.md

@@ -48,6 +48,10 @@ This repo contains GPTQ model files for [Alpin's Goliath 120B](https://huggingfa
 
 Multiple GPTQ parameter permutations are provided; see Provided Files below for details of the options provided, their parameters, and the software used to create them.
 
+**NOTE**: The 4-bit models have been sharded, as otherwise they cannot be uploaded on HF due to the 50GB file limit. This means they will not work with AutoGPTQ at the time of writing.
+
+They will work fine with ExLlama, TGI, and via Transformers.
+
 These files were quantised using hardware kindly provided by [Massed Compute](https://massedcompute.com/).
 
 <!-- description end -->
@@ -68,13 +72,10 @@ You are a helpful AI assistant.
 
 USER: {prompt}
 ASSISTANT:
-
 ```
 
 <!-- prompt-template end -->
 
-
-
 <!-- README_GPTQ.md-compatible clients start -->
 ## Known compatible clients / servers
 
@@ -112,8 +113,8 @@ Most GPTQ files are made with AutoGPTQ. Mistral models are currently made with T
 
 | Branch | Bits | GS | Act Order | Damp % | GPTQ Dataset | Seq Len | Size | ExLlama | Desc |
 | ------ | ---- | -- | --------- | ------ | ------------ | ------- | ---- | ------- | ---- |
-| [main](https://huggingface.co/TheBloke/goliath-120b-GPTQ/tree/main) | 4 | None | Yes | 0.1 | [wikitext](https://huggingface.co/datasets/wikitext/viewer/wikitext-2-raw-v1) | 4096 | 9.99 GB | Yes | 4-bit, with Act Order. No group size, to lower VRAM requirements. | 
-| [gptq-4bit-128g-actorder_True](https://huggingface.co/TheBloke/goliath-120b-GPTQ/tree/gptq-4bit-128g-actorder_True) | 4 | 128 | Yes | 0.1 | [wikitext](https://huggingface.co/datasets/wikitext/viewer/wikitext-2-raw-v1) | 4096 | 9.95 GB | Yes | 4-bit, with Act Order and group size 128g. Uses even less VRAM than 64g, but with slightly lower accuracy. | 
+| [main](https://huggingface.co/TheBloke/goliath-120b-GPTQ/tree/main) | 4 | None | Yes | 0.1 | [wikitext](https://huggingface.co/datasets/wikitext/viewer/wikitext-2-raw-v1) | 4096 | 58.36 GB | Yes | 4-bit, with Act Order. No group size, to lower VRAM requirements. | 
+| [gptq-4bit-128g-actorder_True](https://huggingface.co/TheBloke/goliath-120b-GPTQ/tree/gptq-4bit-128g-actorder_True) | 4 | 128 | Yes | 0.1 | [wikitext](https://huggingface.co/datasets/wikitext/viewer/wikitext-2-raw-v1) | 4096 | 60.56 GB | Yes | 4-bit, with Act Order and group size 128g. Uses even less VRAM than 64g, but with slightly lower accuracy. | 
 | [gptq-3bit--1g-actorder_True](https://huggingface.co/TheBloke/goliath-120b-GPTQ/tree/gptq-3bit--1g-actorder_True) | 3 | None | Yes | 0.1 | [wikitext](https://huggingface.co/datasets/wikitext/viewer/wikitext-2-raw-v1) | 4096 | 45.11 GB | No | 3-bit, with Act Order and no group size. Lowest possible VRAM requirements. May be lower quality than 3-bit 128g. | 
 | [gptq-3bit-128g-actorder_True](https://huggingface.co/TheBloke/goliath-120b-GPTQ/tree/gptq-3bit-128g-actorder_True) | 3 | 128 | Yes | 0.1 | [wikitext](https://huggingface.co/datasets/wikitext/viewer/wikitext-2-raw-v1) | 4096 | 47.25 GB | No | 3-bit, with group size 128g and act-order. Higher quality than 128g-False. |