Chocolatine-14B-Instruct-4k-DPO

DPO fine-tuned of microsoft/Phi-3-medium-4k-instruct (14B params)
using the jpacifico/french-orca-dpo-pairs-revised rlhf dataset.
Training in French also improves the model in English, surpassing the performances of its base model (MMLU).
Window context = 4k tokens

Benchmarks

Chocolatine-14B is the best-performing < 30B model in terms of MMLU-PRO on the OpenLLM Leaderboard (august 2024)

image/png

Metric Value
Avg. 29.83
IFEval (0-Shot) 46.89
BBH (3-Shot) 48.02
MATH Lvl 5 (4-Shot) 14.88
GPQA (0-shot) 12.19
MuSR (0-shot) 15.15
MMLU-PRO (5-shot) 41.82

MT-Bench-French

Chocolatine-14B-Instruct-4k-DPO is outperforming GPT-3.5-Turbo and Phi-3-medium-4k-instruct on
MT-Bench-French, used with multilingual-mt-bench

########## First turn ##########
                                           score
model                               turn        
Chocolatine-14B-Instruct-4k-DPO     1     8.6375
Phi-3-medium-4k-instruct            1     8.2250
gpt-3.5-turbo                       1     8.1375
Chocolatine-3B-Instruct-DPO-Revised 1     7.9875
Daredevil-8B                        1     7.8875
Chocolatine-3B-Instruct-DPO-v1.0    1     7.6875
NeuralDaredevil-8B-abliterated      1     7.6250
Phi-3-mini-4k-instruct              1     7.2125
Meta-Llama-3-8B-Instruct            1     7.1625
vigostral-7b-chat                   1     6.7875
Mistral-7B-Instruct-v0.3            1     6.7500
Mistral-7B-Instruct-v0.2            1     6.2875

########## Second turn ##########
                                             score
model                               turn          
Chocolatine-3B-Instruct-DPO-Revised 2     7.937500
Phi-3-medium-4k-instruct            2     7.750000
Chocolatine-14B-Instruct-4k-DPO     2     7.737500
gpt-3.5-turbo                       2     7.679167
Chocolatine-3B-Instruct-DPO-v1.0    2     7.612500
NeuralDaredevil-8B-abliterated      2     7.125000
Daredevil-8B                        2     7.087500
Meta-Llama-3-8B-Instruct            2     6.800000
Mistral-7B-Instruct-v0.2            2     6.512500
Mistral-7B-Instruct-v0.3            2     6.500000
Phi-3-mini-4k-instruct              2     6.487500
vigostral-7b-chat                   2     6.162500

########## Average ##########
                                        score
model                                        
Chocolatine-14B-Instruct-4k-DPO      8.187500
Phi-3-medium-4k-instruct             7.987500
Chocolatine-3B-Instruct-DPO-Revised  7.962500
gpt-3.5-turbo                        7.908333
Chocolatine-3B-Instruct-DPO-v1.0     7.650000
Daredevil-8B                         7.487500
NeuralDaredevil-8B-abliterated       7.375000
Meta-Llama-3-8B-Instruct             6.981250
Phi-3-mini-4k-instruct               6.850000
Mistral-7B-Instruct-v0.3             6.625000
vigostral-7b-chat                    6.475000
Mistral-7B-Instruct-v0.2             6.400000

Usage

You can run this model using my Colab notebook

You can also run Chocolatine using the following code:

import transformers
from transformers import AutoTokenizer

# Format prompt
message = [
    {"role": "system", "content": "You are a helpful assistant chatbot."},
    {"role": "user", "content": "What is a Large Language Model?"}
]
tokenizer = AutoTokenizer.from_pretrained(new_model)
prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)

# Create pipeline
pipeline = transformers.pipeline(
    "text-generation",
    model=new_model,
    tokenizer=tokenizer
)

# Generate text
sequences = pipeline(
    prompt,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    num_return_sequences=1,
    max_length=200,
)
print(sequences[0]['generated_text'])

Limitations

The Chocolatine model is a quick demonstration that a base model can be easily fine-tuned to achieve compelling performance.
It does not have any moderation mechanism.

  • Developed by: Jonathan Pacifico, 2024
  • Model type: LLM
  • Language(s) (NLP): French, English
  • License: MIT
Downloads last month
5,109
Safetensors
Model size
14B params
Tensor type
FP16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for jpacifico/Chocolatine-14B-Instruct-4k-DPO

Merges
5 models
Quantizations
3 models

Dataset used to train jpacifico/Chocolatine-14B-Instruct-4k-DPO

Space using jpacifico/Chocolatine-14B-Instruct-4k-DPO 1

Collection including jpacifico/Chocolatine-14B-Instruct-4k-DPO