QuantFactory Banner

QuantFactory/Llama3.1-8B-Esper2-GGUF

This is quantized version of ValiantLabs/Llama3.1-8B-Esper2 created using llama.cpp

Original Model Card

image/jpeg

Esper 2 is a DevOps and cloud architecture code specialist built on Llama 3.1 8b.

  • Expertise-driven, an AI assistant focused on AWS, Azure, GCP, Terraform, Dockerfiles, pipelines, shell scripts and more!
  • Real world problem solving and high quality code instruct performance within the Llama 3.1 Instruct chat format
  • Finetuned on synthetic DevOps-instruct and code-instruct data generated with Llama 3.1 405b.
  • Overall chat performance supplemented with generalist chat data.

Try our code-instruct AI assistant Enigma!

Version

This is the 2024-10-02 release of Esper 2 for Llama 3.1 8b.

Esper 2 is now available for Llama 3.2 3b!

Esper 2 will be coming to more model sizes soon :)

Prompting Guide

Esper 2 uses the Llama 3.1 Instruct prompt format. The example script below can be used as a starting point for general chat:

import transformers
import torch

model_id = "ValiantLabs/Llama3.1-8B-Esper2"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are an AI assistant."},
    {"role": "user", "content": "Hi, how do I optimize the size of a Docker image?"}
]

outputs = pipeline(
    messages,
    max_new_tokens=2048,
)

print(outputs[0]["generated_text"][-1])

The Model

Esper 2 is built on top of Llama 3.1 8b Instruct, improving performance through high quality DevOps, code, and chat data in Llama 3.1 Instruct prompt style.

Our current version of Esper 2 is trained on DevOps data from sequelbox/Titanium, supplemented by code-instruct data from sequelbox/Tachibana and general chat data from sequelbox/Supernova.

image/jpeg

Esper 2 is created by Valiant Labs.

Check out our HuggingFace page for Shining Valiant 2 Enigma, and our other Build Tools models for creators!

Follow us on X for updates on our models!

We care about open source. For everyone to use.

We encourage others to finetune further from our models.

Downloads last month
41
GGUF
Model size
8.03B params
Architecture
llama

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Examples
Unable to determine this model's library. Check the docs .

Model tree for QuantFactory/Llama3.1-8B-Esper2-GGUF

Quantized
(301)
this model

Datasets used to train QuantFactory/Llama3.1-8B-Esper2-GGUF

Evaluation results