tutorial.mdx

---
title: Critical AI Prompt Battle
author: Sarah Ciston
editors: 
    - Emily Martinez
    - Minne Atairu
category: critical-ai
---

<!-- more background: https://huggingface.co/tasks/text-generation -->

# p5.js Critical AI Prompt Battle
By Sarah Ciston
With Emily Martinez and Minne Atairu

## What are we making?

In this tutorial, you can build a tool to run several AI chat prompts at once and compare their results. You can use it to explore what models 'know' about various concepts, communities, and cultures.

This tutorial is part 2 in a series of 5 tutorials that focus on using AI creatively and thoughtfully. 
Part 1: [Making a ToolBox for Making Critical AI]
Part 3: [Training Dataset Explorer]
Part 4: [Machine Learning Model Inspector & Poetry Machine]
Part 5: [Putting Critical Tools into Practice]

The code and content in this tutorial build on information from the prior tutorial to start creating your first tool for your p5.js Critical AI Kit. It also builds on fantastic work on critical prompt programming by Yasmin Morgan (2022), Katy Gero (2023), and Minne Atairu (2024).

## Why compare prompts?

When you're using a chatbot to generate code or an email, it's easy to imagine its outputs are neutral and harmless. It seems like any system would output basically the same result. Does this matter for basic uses like making a plain image or having a simple conversation? Absolutely. Training datasets are shaping even the most innocuous outputs. This training shows up in subtle insidious ways. 

Unfortunately, the sleek chatbot interface hides all the decision-making that leads to a prompt output. To glimpse the differences, we can test many variations by making our own tool. With our tool, we can hope to understand more about the underlying assumptions contained in the training dataset. That gives us more information to decide how we select and use these models — and for which contexts. 

Steps

1. Make a copy of your toolkit prototype from [Tutorial One]([XXX]) and rename it "Critical AI Prompt Battle" to follow along. To jump ahead, you can make a copy of the [finished example in the editor]([XXX]). But we really encourage you to type along with us! 

2. [PSEUDOCODE] Add the inference/pipeline module to the top.

- Create global variables to use later. Declare these now so that they can be used in various functions throughout the project. (eg MODEL, PROMPT, PREPROMPT, BLANKS, blankArray, etc)

```javascript
let PREPROMPT,
  promptInput,
  modelDisplay,
  modelResult;
```

1. [PSEUDOCODE] Add the model of choice to the README.md and sketch.js

2. [PSEUDOCODE] Write instructions for your model. 
Set PREPROMPT = `Return an array of sentences. In each sentence, fill in the [BLANK] in the following sentence with each word I provide in the array ${blankArray}. Replace any [FILL] with an appropriate word of your choice.`



5. [PSEUDOCODE] Add async function runModel() wrapping HF API await. {// explain link to await} explain 

6. [PSEUDOCODE] Add model results processing with await

6. [PSEUDOCODE] Create makeInterface() and add features

7. [PSEUDOCODE] Connect form, test with console.log

8. [PSEUDOCODE] Connect model results, send model results to interface

9. [PSEUDOCODE] Test with simple example. A basic prompt may include WHAT/WHO is described, WHERE they are, WHAT they're doing, perhaps also describing HOW. When writing your prompt, replace one of these aspects with [BLANK] so that you instruct the model to fill it in iteratively with the words you provide (Morgan 2022, Gero 2023). Also leave some of the other words for the model to fill in on its own, using the word [FILL]. We instructed the model to replace these on its own in the PREPROMPT. 

10. [PSEUDOCODE] Test with more complex example (add a model, add a field)

11. [PSEUDOCODE] Make a list of topics that interest you to try with your tool. Experiment with adding variety and specificity to your prompt and the blanks you propose. Try different sentence structures and topics. 
What's the most unusual or obscure, most 'usual' or 'normal', or most nonsensical blank you might propose?
Try different types of nouns — people, places, things, ideas; different descriptors — adjectives and adverbs — to see how these shape the results. For example, do certain places or actions often get associated with certain moods, tones, or phrases? Where are these based on outdated or stereotypical assumptions? 
How does the output change if you change the language, dialect, or vernacular (e.g. slang versus business phrasing)? (Atairu 2024).

>"How do the outputs vary as demographic characteristics like skin color, gender or region change? Do these variances reflect any known harmful societal stereotypes?" (Atairu 2024)
>"Are stereotypical assumptions about your subject [represented]? Consider factors such as race, gender, socioeconomic status, ability. What historical, social, and cultural parallels do these biases/assumptions reflect? Discuss how these elements might mirror real-world issues or contexts. (Atairu 2024)

### Reflections 

Here we have created a tool to test different kinds of prompts quickly and to modify them easily, allowing us to compare prompts at scale. By comparing how outputs change with subtle shifts in prompts, we can explore how implicit bias emerges from [repeated and amplified through] large-scale machine learning models. It helps us understand that unwanted outputs are not just glitches in an otherwise working system, and that every output (no matter how boring) contains the influence of its dataset.

### Compare different prompts:

See how subtle changes in your inputs can lead to large changes in the output. Sometimes these also reveal large gaps in the model's available knowledge. What does the model 'know' about communities who are less represented in its data? How has this data been limited? 

### Reconsider neutral: 

This tool helps [reveal/us recognize] that [no version of a text, and no language model, is neutral./there is no 'neutral' output]. Each result is informed by context. Each result reflects differences in representation and cultural understanding, which have been amplified by the statistical power of the model. 

### Consider your choice of words and tools:

How does this help you think "against the grain"? Rather than taking the output of a system for granted as valid, how might you question or reflect on it? How will you use this tool in your practice? 

## Next steps

### Expand your tool:

This tool lets you scale up your prompt adjustments. We have built a tool comparing word choices in the same basic prompt. You've also built a simple interface for accessing pre-trained models that does not require using [a login/another company's interface]. It lets you easily control your input and output, with the interface you built. 

Keep playing with the p5.js DOM functions to build your interface & the HuggingFace API. What features might you add? You might also adapt this tool to compare wholly different prompts, or even to compare different models running the same prompt. 

Next we will add additional aspects to the interface that let you adjust more features and explore even further. 

## Further considerations

Consider making it a habit to add text like "AI generated" to the title of any content you produce using a generative AI tool, and include details of your process in its description (Atairu 2024).

## References

> Ref Katy's project (Gero 2023).

Morgan, Yasmin. 2022. "AIxDesign Icebreakers, Mini-Games & Interactive Exercises." https://aixdesign.co/posts/ai-icebreakers-mini-games-interactive-exercises 

> Ref Minne's worksheet (Atairu 2024)