README.md

---
license: mit
tags:
- generated_from_trainer
model-index:
- name: GPT2-Medium-Alpaca-355m
  results: []
datasets:
- tatsu-lab/alpaca
widget:
- text: |-
    ### Instruction:
    Make a list of items necessary to bring on a trip to a volcano.

    ### Response:
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# GPT2-Medium-Alpaca-355m

This model is a fine-tuned version of [gpt2-medium](https://huggingface.co/gpt2-medium) on the [tatsu-lab/alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) dataset. This is a test to check if the concept of Databrick's [databricks/dolly-v1-6b](https://huggingface.co/databricks/dolly-v1-6b) can be applied to smaller models.

## Model description

This is an LLM trained to follow instructions, similar to ChatGPT, except much much smaller.

## Sample Use

```python
from transformers import pipeline

generator = pipeline('text-generation', model='linkanjarad/GPT2-Medium-Alpaca-355m')

prompt = '''### Instruction:
Who is the first person on the moon?

### Response:'''

output = generator(prompt, do_sample=True, min_length=10, max_length=50, temperature=0.5)
print(output[0]['generated_text'])
```

## Intended uses & limitations

The model works best when given input in the format of the dataset it was trained on. Here is an example of the format of the [tatsu-lab/alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) dataset:

```
### Instruction:
What are the three primary colors?

### Response:
The three primary colors are red, blue, and yellow.
```

Here are a few sample model outputs:

```
### Instruction:
What is the most common type of sugar found in baked goods?

### Response:
The most common type of sugar found in baked goods are high fructose corn syrup, sucrose, high-fructose corn syrup, and artificial sweeteners. Sucrose is the highest-fructose sugar, and is mainly found in certain types of bakery products, such as cakes, cookies, and pastries. HFCS is made from corn syrup and is typically found in processed products
```

```
### Instruction:
List some steps to take for a project plan regarding the implementation of a food drive.

### Response:
To implement a food drive, it is important to set concrete goals and objectives. These goals and objectives should include:
1. Collecting the necessary supplies to provide the hungry
2. Setting up the food banks to store the produce
3. Organizing distribution of the produce
4. Reducing waste and reducing waste disposal
5. Monitoring the progress of the plan and ensuring it is executed correctly
6. Leveraging the existing community, resources, and resources to implement the plan
7. Monitoring the effectiveness of the plan and providing feedback on progress
8. Establishing and reviewing financial milestones (when required)
9. Creating and running training sessions for staff and volunteers
10. Responding to feedback from customers and visitors
11. Establishing guidelines and standards in the program
12. Keeping up to date with the latest nutrition information and trends
13. Developing sustainability plans and strategies to maintain a healthy kitchen environment
14. Developing a business plan for marketing and selling the produce, ingredients, and products
15. Responding to customer feedback and implementing them
16. Evaluating customer satisfaction levels and improving the program, especially when the program is not effective
17. Having regular meetings with the board and stakeholders to discuss progress and recommendations
18. Reviewing the program and taking steps to improve it as time goes on
```

```
### Instruction:
Generate a short horror story about evil teddy bears.

### Response:
A young girl named Elizabeth wandered down a dark and mysterious alley in London. At the end of the alley, her nose ran into a strange-looking teddy bear. It was covered in dark fur and had a dark and unsettling smell. Elizabeth slowly noticed that he was shaking and didn't want to sleep. He seemed extremely tense and scared and she decided to take him in her arms and bring him home. Despite all the fear and fear this strange and scary bear held, Elizabeth was determined to help him. She went through the alley as quickly and quietly as possible, avoiding people and animals alike to avoid getting lost. She eventually found her way back home and was deeply moved by the bear's kindness. She wanted to make sure that this strange bear that she encountered was still there whenever she needed it.

Eventually Elizabeth was able to convince the bear to stay, and he kept him safe by staying with her without saying a word. Elizabeth thanked her for the gift and promised to help to make sure he always had safe haven, and the bear was thankful and soon accepted her offer.

Elizabeth made sure to take this teddy to see him the next time she visited London and he quickly became something of a friend when she returned. He has since become so fond of the small bear that he always wears around their home and has even been named after Elizabeth.
```

```
### Instruction:
Who discovered gravity?

### Response:
The first person to discover gravity was an Italian mathematician, Leonardo da Vinci. He coined the term 'gravity' in 1584. Da Vinci's discovery was so revolutionary that it was eventually used in textbooks by 19th century geologists and astronomers. Since then, scientists have been discovering numerous other forces, like electromagnetic waves, magnetism, gravity, and friction. In recent years, numerous scientists have used the term 'gravity
```

```
### Instruction:
Explain the process of how a star is formed.

### Response:
A star is a star that is formed from a gas and dust cloud of gas and dust, known as a disk, that is surrounded by a dense outer disk. The outer disk surrounds an energy-rich, liquid-filled outer crust. The density of the disk is very high, increasing the amount of material it contains, which creates the gravitational pull of the planet within it. The outer crust is filled with a high degree of material-rich metals, carbon, and iron, which are the building blocks for the stars and planets in our solar system. When the disk is fully formed, the metal-rich layers of the outer disk are ejected outwards into space, creating a star. Stars are the primary means of transporting matter from the outer disk to the core of the planet. The core of a planet is composed of layers of mostly metals, with some oxygen in the center of the core
```

## Training Procedure

The model was trained with Huggingface's 🤗 Trainer for about half an epoch.

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 128
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.15
- num_epochs: 1

### Framework versions

- Transformers 4.27.4
- Pytorch 1.13.0
- Datasets 2.1.0
- Tokenizers 0.13.2