---
tags:
- generated_from_trainer
model-index:
- name: fiction_story_generator
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# GPT-Neo for Fiction Story Generation

This model is a fine-tuned version of EleutherAI's GPT-Neo-125M model, optimized for generating fiction stories. 
It has been trained on the dataset available at https://github.com/facebookresearch/fairseq/tree/main/examples/stories.


## Model description

- Model Name: GPT-Neo-Fiction
- Fine-Tuned By: Tin Kanjovsky/Tincando, Darko Etinger
- Model Version: 1.0

## Intended uses & limitations

The model is designed for creative fiction story generation. It can be used for various purposes, including but not limited to:

- Storytelling: Generate engaging and imaginative fiction stories.
- Content Generation: Create content for blogs, websites, or other media with a storytelling element.
- Creative Writing: Assist authors and writers in brainstorming ideas and expanding narratives.

## Model Performance

- Training Data: The model was trained on a diverse dataset of fiction stories.
- Evaluation Metrics: Performance metrics, such as perplexity or BLEU scores, may vary depending on the specific task and dataset.

## Limitations

- Content Quality: While GPT-Neo-Fiction can generate creative stories, the quality and coherence of the output may vary, and it may occasionally produce nonsensical or inappropriate content.
- Bias: The model may exhibit biases present in the training data, and it is essential to be cautious when using it for sensitive topics or content.
- Length of Output: The model may generate text with variable length, and it may not always produce the desired length of output.
- Fine-Tuning Data: The quality of generated stories is dependent on the quality and diversity of the fine-tuning dataset.

## Usage

```
from transformers import GPTNeoForCausalLM, GPT2Tokenizer

tokenizer = GPT2Tokenizer.from_pretrained(Tincando/fiction_story_generator)
model = GPTNeoForCausalLM.from_pretrained(Tincando/fiction_story_generator)

# Generate a fiction story
input_prompt = "[WP] I can't believe I died the same way twice."
input_ids = tokenizer(input_prompt, add_special_tokens=False, return_tensors="pt").input_ids

output = model.generate(input_ids,
        max_length=300,
        temperature=0.9,
        top_k=2,
        top_p=0.9,
        repetition_penalty=1.2,
        do_sample=True,
        num_return_sequences=2
)

generated_story = tokenizer.batch_decode(output,clean_up_tokenization_spaces=True)[0]
print(generated_story)
```

## Ethical Considerations

When using this model, consider the following ethical guidelines:

- Content Moderation: Implement content moderation to ensure that the generated stories do not violate guidelines or community standards.
- Bias and Fairness: Be aware of potential biases in the model's output and take steps to mitigate them.
- Privacy: Avoid using personal or sensitive information as input prompts.
- Legal Compliance: Ensure that the generated content complies with copyright and intellectual property laws.

## Citation
If you use GPT-Neo-Fiction in your work, please consider citing the original GPT-Neo model and the dataset used for fine-tuning:

- [GPT-Neo Paper](https://github.com/EleutherAI/gpt-neo)
- [Fairseq Repository](https://github.com/facebookresearch/fairseq/tree/main/examples/stories)
- [Hierarchical Neural Story Generation](https://arxiv.org/abs/1805.04833)

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step   | Validation Loss |
|:-------------:|:-----:|:------:|:---------------:|
| 3.0842        | 1.0   | 34075  | 3.1408          |
| 3.0026        | 2.0   | 68150  | 3.1275          |
| 2.9344        | 3.0   | 102225 | 3.1270          |
| 2.8932        | 4.0   | 136300 | 3.1306          |
| 2.8517        | 5.0   | 170375 | 3.1357          |


### Framework versions

- Transformers 4.28.0
- Pytorch 1.12.1+cu116
- Datasets 2.4.0
- Tokenizers 0.12.1