dzhuz/gpt2-art-description

GPT-2 Fine-Tuned on Art Museum Books

Model Description

This model is a fine-tuned version of GPT-2, adapted to generate text inspired by a self-made dataset of various art museum books. It specializes in creating detailed, context-aware, and stylistically rich descriptions relevant to art and museum contexts.

Training Details

• Base Model: GPT-2
• Dataset: Custom-built dataset containing curated texts from art museum books.
• Steps: Trained for 29,900 steps over ~2 hours.
• Hardware: Google Colab with NVIDIA T4 GPU.
• Hyperparameters:
  • Epochs: 5
  • Batch Size: 8
  • Mixed Precision (fp16): Enabled
  • Save Steps: Every 500 steps
  • Logging Steps: Every 100 steps
  • Evaluation Strategy: None

Training Script

The model was fine-tuned using the transformers library with the following configuration:

from transformers import TrainingArguments, Trainer
from transformers import DataCollatorForLanguageModeling

output_dir = "/content/drive/MyDrive/gpt2-art"
training_args = TrainingArguments(
    output_dir=output_dir,
    overwrite_output_dir=True,
    num_train_epochs=5,
    per_device_train_batch_size=8,
    save_steps=500,
    save_total_limit=2,
    logging_steps=100,
    evaluation_strategy="no",
    fp16=True,
)

data_collator = DataCollatorForLanguageModeling(
    tokenizer=tokenizer,
    mlm=False,
)