Manticore-13b-Landmark-GPTQ

Key Features

Landmark Attention
Large Context Size (~18k)
4-Bit Quantization

Composition

Manticore-13b-Landmark-GPTQ is a 4-bit quantization of:

Manticore-13B-Landmark

Using Oobabooga

Trust Remote Code - (Enabled)
Add the bos_token to the beginning of prompts - (Disabled)
Truncate the prompt up to this length - (Increased)

Landmark Training Code

See GitHub for the training code.

Downloads last month: 16

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.