You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Llama 3.1-8B Instruct African-Ultrachat Quantize

  • Developed by: vutuka
  • License: apache-2.0
  • Finetuned from model : meta-llama/meta-llama-3.1-8b-instruct
  • Max Content Length : 8192
  • Max Steps : 800
  • Training Time : 02h-22min-08s
  • Setup :
    • 1 x RTX A6000
    • 16 vCPU
    • 58 GB RAM
    • 150 GB Storage

Tokenizer & Chat Format

from unsloth.chat_templates import get_chat_template

tokenizer = get_chat_template(
    tokenizer,
    chat_template = "llama-3", # Supports zephyr, chatml, mistral, llama, alpaca, vicuna, vicuna_old, unsloth
    mapping={
        "role": "role",
        "content": "content",
        "user": "",
        "assistant": "",
    }
)

def formatting_prompts_func(examples):
    convos = examples["messages"]
    texts = [tokenizer.apply_chat_template(convo, tokenize = False, add_generation_prompt = False) for convo in convos]
    return { "text" : texts, }
pass

Trainer

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = shuffled_dataset,
    dataset_text_field = "text",
    max_seq_length = max_seq_length,
    dataset_num_proc = 2,
    packing = False, # Can make training 5x faster for short sequences.
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 4,
        warmup_steps = 5,
        max_steps = 800,
        do_eval=True,
        learning_rate = 3e-4,
        log_level="debug",
        #fp16 = not is_bfloat16_supported(),
        bf16 = True,
        logging_steps = 10,
        optim = "adamw_8bit",
        weight_decay = 0.01,
        lr_scheduler_type = "linear",
        seed = 3407,
        output_dir = "outputs",
        report_to='wandb',
        warmup_ratio=0.3,
    ),
)

Inference with Llama CPP

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

Downloads last month
117
GGUF
Model size
8.03B params
Architecture
llama

4-bit

5-bit

8-bit

16-bit

Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for vutuka/Llama-3.1-8B-Instruct-African-Ultrachat-GGUF

Quantized
(301)
this model