DCLM-7B-Chat

This is a fine-tuned version of the DCLM-7B baseline model trained for chat completions.

Quick start

To use the model, open_lm must first be installed:

pip install git+https://github.com/mlfoundations/open_lm.git

Then simply load the model and generate responses:

from open_lm.hf import *
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
)


model = AutoModelForCausalLM.from_pretrained("mathewhe/DCLM-7B-Chat")
tokenizer = AutoTokenizer.from_pretrained("mathewhe/DCLM-7B-Chat")

messages = [
    {"role": "user", "content": "What is an LLM?"},
]

inputs = tokenizer.apply_chat_template(messages)

print(tokenizer.decode(model.generate(**inputs)[0]))

Chat template

This model uses the following chat template and does not support a separate system prompt:

<|endoftext|>[INST] <user-message> [/INST][ASST] <llm-response> [/ASST]<|endoftext|>

The included tokenizer will correctly format messages, so you should not have to manually format the input text.

Instead, use the tokenizer's apply_chat_template() method on a list of messages. Each message should be a dict with two keys:

"role": Either "user" or "assistant".
"content": The message to include.

For example:

messages = [
    {"role": "user", "content": "Solve for x: 3x=4"},
    {"role": "assistant", "content": "3x=4\n(3x)/3=(4)/3\nx=4/3"},
    {"role": "user", "content": "Please explain your work."},
]

See the example code in the included chat_class.py module for more details.