Hugging Face's logo

sartajbhuvaji
/
facebook-opt-350m-chat

Text Generation
Transformers
Safetensors
opt
trl
sft
chat
text-generation-inference
Inference Endpoints
Model card Files Community
facebook-opt-350m-chat / README.md
sartajbhuvaji's picture
sartajbhuvaji
Update README.md
8e824be verified
preview code
|
raw
history blame contribute delete
3.74 kB
metadata
library_name: transformers
tags:
  - trl
  - sft
  - chat
license: apache-2.0
datasets:
  - lucasmccabe-lmi/CodeAlpaca-20k
base_model:
  - facebook/opt-350m
pipeline_tag: text-generation

Model Card for Finetuned OPT-350M Chatbot Model

Model Details

Model Description

This is a chat fine-tuned version of facebook/opt-350m, designed to provide chatbot-like responses using instruction fine-tuning techniques. The goal of this tuning was to to convert a Base Model to Chat Model using Instruction Finetuning.

  • Developed by: Sartaj
  • Finetuned from model: facebook/opt-350m
  • Language(s): English
  • License: apache-2.0
  • Framework: Hugging Face Transformers

Model Sources

Uses

Model can be used to generate basic code and further finetuned to refine code generation.

Usage 

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "sartajbhuvaji/facebook-opt-350m-chat"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)

def generate_response(question):
    input_prompt = f"### Question: {question}\n ### Answer:"
    inputs = tokenizer(input_prompt, return_tensors="pt").to(device)

    # Generate output using the model
    outputs = model.generate(
        inputs["input_ids"],
        max_length=500, 
        num_beams=5,  
        temperature=0.7, 
        eos_token_id=tokenizer.eos_token_id,
        early_stopping=True,
    )

    generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return generated_text

question = "Write a Python program to add two numbers."
response = generate_response(question)
print(response)

'''
### Question: Write a Python program to add two numbers.
 ### Answer: def add_two_numbers(a, b):
    return a + b 
'''

Downstream Use

  • Code Geneation
  • Fine Tuning

Training Details

Training Data

Training Procedure

  • Full Model Finetune
  • Epochs : 3

Preprocessing

  • Pre Processed data to follow template: ### Question: {quesion}\n ### Answer: {ansewer} {tokenizer.eos_token}
def formatting_prompts_func(example):
    output_texts = []
    for i in range(len(example['instruction'])):
        text = f"### Question: {example['instruction'][i]}\n ### Answer: {example['output'][i]} {tokenizer.eos_token}"
        output_texts.append(text)
    return output_texts

Training Loss

image/png

Trainer

  • global_step: 7509
  • training_loss: 0.9127310856885068
  • train_runtime: 2485.7984
  • train_samples_per_second: 24.164
  • train_steps_per_second: 3.021
  • total_flos: 2.939309944327373e+16
  • train_loss: 0.9127310856885068
  • epoch: 3.0

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

  • Hardware Type: NVIDIA A100 40 GB
  • Hours used: ~10
  • Cloud Provider: jetstream2
  • Compute Region: USA
  • Carbon Emitted: 2.24 Kg