README.md

---
library_name: transformers
tags:
- trl
- sft
- chat
license: apache-2.0
datasets:
- lucasmccabe-lmi/CodeAlpaca-20k
base_model:
- facebook/opt-350m
pipeline_tag: text-generation
---

# Model Card for Finetuned OPT-350M Chatbot Model

## Model Details

### Model Description

This is a chat fine-tuned version of `facebook/opt-350m`, designed to provide chatbot-like responses using instruction fine-tuning techniques.
The goal of this tuning was to to convert a Base Model to Chat Model using Instruction Finetuning.

- **Developed by:** [Sartaj](https://huggingface.co/sartajbhuvaji)
- **Finetuned from model:** `facebook/opt-350m`
- **Language(s):** English
- **License:** apache-2.0
- **Framework:** Hugging Face Transformers


### Model Sources 

- **Repository:** [facebook/opt-350m](https://huggingface.co/facebook/opt-350m)
- **Paper:** [paper](https://arxiv.org/pdf/2205.01068)

## Uses

Model can be used to generate basic code and further finetuned to refine code generation.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "sartajbhuvaji/facebook-opt-350m-chat"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)

def generate_response(question):
    input_prompt = f"### Question: {question}\n ### Answer:"
    inputs = tokenizer(input_prompt, return_tensors="pt").to(device)

    # Generate output using the model
    outputs = model.generate(
        inputs["input_ids"],
        max_length=500, 
        num_beams=5,  
        temperature=0.7, 
        eos_token_id=tokenizer.eos_token_id,
        early_stopping=True,
    )

    generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return generated_text

question = "Write a Python program to add two numbers."
response = generate_response(question)
print(response)

'''
### Question: Write a Python program to add two numbers.
 ### Answer: def add_two_numbers(a, b):
    return a + b 
'''

```

### Downstream Use 

- Code Geneation
- Fine Tuning

## Training Details

### Training Data

- **Dataset** : [lucasmccabe-lmi/CodeAlpaca-20k](https://huggingface.co/datasets/lucasmccabe-lmi/CodeAlpaca-20k)
- **Total Training Tokens** : 2,202,939

### Training Procedure

- Full Model Finetune
- Epochs : 3

#### Preprocessing
- Pre Processed data to follow template: ### Question: {quesion}\n ### Answer: {ansewer} {tokenizer.eos_token}
  
``` python
def formatting_prompts_func(example):
    output_texts = []
    for i in range(len(example['instruction'])):
        text = f"### Question: {example['instruction'][i]}\n ### Answer: {example['output'][i]} {tokenizer.eos_token}"
        output_texts.append(text)
    return output_texts
```


#### Training Loss
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6354695712edd0ed5dc46b04/SaJTUSQvBmT6uH5gAVQQT.png)

## Trainer 
- global_step: 7509
- training_loss: 0.9127310856885068
- train_runtime: 2485.7984
- train_samples_per_second: 24.164
- train_steps_per_second: 3.021
- total_flos: 2.939309944327373e+16
- train_loss: 0.9127310856885068
- epoch: 3.0


## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** NVIDIA A100 40 GB
- **Hours used:**  ~10
- **Cloud Provider:** jetstream2
- **Compute Region:** USA
- **Carbon Emitted:** 2.24 Kg