Wizard-Vicuna-13B-Uncensored-SpQR
Overview
This model is an SpQR 4-bit quantization of the original Wizard-Vicuna-13B-Uncensored-HF
Quantization Specifications
- Quantization: 4-bit, group size of 16, per-channel with scale and zero-point of 3 bits.
- Outliers: Threshold set at 0.2.
- Permutation Order:
act_order. - Dampening: Set at 1e0.
- Sampling: 128 samples.
- Logging: Via Weights & Biases.
Evaluation Metrics
The following perplexity scores were obtained on various datasets:
| Dataset | Perplexity |
|---|---|
| c4 | 7.354 |
| wikitext2 | 5.685 |
| ptb | 20.822 |
- Downloads last month
- 9
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
the model is not deployed on the HF Inference API.
Evaluation results
- perplexity on C4self-reported7.354
- perplexity on WikiText-2self-reported5.685
- perplexity on PTBself-reported20.822