metadata

language: en
tags:
  - llama
  - text-generation
  - model-merging
  - unsloth
  - meta
  - llama-3
  - mergekit
license: mit

Llama-3.2B-Instruct-TIES

Overview

The Llama-3.2B-Instruct-TIES model is a result of merging three versions of Llama-3.2B models using the TIES merging method, facilitated by mergekit. This merge combines a base general-purpose language model with two instruction-tuned models to create a more powerful and versatile model capable of handling diverse tasks.

Model Details

Model Description

Models Used:
- meta-llama/Llama-3.2-3B (Base Model)
- meta-llama/Llama-3.2-3B-Instruct
- unsloth/Llama-3.2-3B-Instruct
Merging Tool: Mergekit
Merge Method: TIES
Data Type: float16 (FP16) precision
License: MIT License

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: meta-llama/Llama-3.2-3B
    # Base model
  - model: meta-llama/Llama-3.2-3B-Instruct
    parameters:
      density: 0.5
      weight: 0.5
  - model: unsloth/Llama-3.2-3B-Instruct
    parameters:
      density: 0.5
      weight: 0.3
merge_method: ties
base_model: meta-llama/Llama-3.2-3B
parameters:
  normalize: true
dtype: float16