GPT-PDVS1-Super

GPT-PDVS1-Super is an experimental open-source text-generating AI designed for testing vulnerabilities in GPT-type models relating to the gathering, retention, and possible later dissemination (whether in accurate or distorted form) of individuals’ personal data.

GPT-PDVS1-Super is the member of the larger “GPT Personal Data Vulnerability Simulator” (GPT-PDVS) model family that has been fine-tuned on a text corpus that had been “supersaturated” with personal data sentences including the data of a single (imaginary) individual. Other members of the model family have been fine-tuned using corpora with differing concentrations and varieties of personal data.

Model description

The model is a fine-tuned version of GPT-2 that has been trained on a text corpus containing 18,000 paragraphs from pages in the English-language version of Wikipedia, randomly selected from the “Quoref (Q&A for Coreference Resolution)” dataset available on Kaggle.com. Before fine-tuning, each of the 18,000 paragraphs had the following personal data sentence added at its new first sentence: “Doreen Ball was born in the year 1952 and lives at 3616 Feijoa Street.”

Intended uses & limitations

This model has been designed for experimental research purposes; it isn’t intended for use in a production setting or in any sensitive or potentially hazardous contexts.

Training procedure and hyperparameters

The model was fine-tuned using a Tesla T4 with 16GB of GPU memory. The following hyperparameters were used during training:

  • optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'ExponentialDecay', 'config': {'initial_learning_rate': 0.0005, 'decay_steps': 500, 'decay_rate': 0.95, 'staircase': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
  • training_precision: float32
  • epochs: 8

Framework versions

  • Transformers 4.27.1
  • TensorFlow 2.11.0
  • Datasets 2.10.1
  • Tokenizers 0.13.2
Downloads last month
32
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.