Week 1 Day 2

Accessing_OpenAI_Like_a_Developer_AIMS_(Assignment_Version).ipynb

@@ -0,0 +1,954 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "zPtFBgj623FS"
+      },
+      "source": [
+        "# Accessing OpenAI Like a Developer\n",
+        "\n",
+        "- 🤝 Breakout Room #1:\n",
+        "  1. Getting Started\n",
+        "  2. Setting Environment Variables\n",
+        "  3. Using the OpenAI Python Library\n",
+        "  4. Prompt Engineering Principles\n",
+        "  5. Testing Your Prompt"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "-Pa34dMvQ6Ai"
+      },
+      "source": [
+        "# How AIM Does Assignments\n",
+        "\n",
+        "If you look at the Table of Contents (accessed through the menu on the left) - you'll see this:\n",
+        "\n",
+        "![image](https://i.imgur.com/I8iDTUO.png)\n",
+        "\n",
+        "Or this if you're in Colab:\n",
+        "\n",
+        "![image](https://i.imgur.com/0rHA1yF.png)\n",
+        "\n",
+        "You'll notice during assignments that we have two following categories:\n",
+        "\n",
+        "1. ❓ - Questions. These will involve...answering questions!\n",
+        "2. 🏗️ - Activities. These will involve writing code, or modifying text.\n",
+        "\n",
+        "In order to receive full marks on the assignment - it is expected you will answer all questions, and complete all activities."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "1w4egfB274VD"
+      },
+      "source": [
+        "## 1. Getting Started\n",
+        "\n",
+        "The first thing we'll do is load the [OpenAI Python Library](https://github.com/openai/openai-python/tree/main)!"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 1,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "23H7TMOM4mfy",
+        "outputId": "698578e4-f787-41d6-fe93-249b8af78b9a"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "Note: you may need to restart the kernel to use updated packages.\n"
+          ]
+        }
+      ],
+      "source": [
+        "%pip install openai -q"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "xKD8XBTVEAOw"
+      },
+      "source": [
+        "## 2. Setting Environment Variables\n",
+        "\n",
+        "As we'll frequently use various endpoints and APIs hosted by others - we'll need to handle our \"secrets\" or API keys very often.\n",
+        "\n",
+        "We'll use the following pattern throughout this bootcamp - but you can use whichever method you're most familiar with."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 2,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "RGU9OMvhEPG0",
+        "outputId": "2609c2ff-e2f5-4464-fb26-38a4fcfe7ea3"
+      },
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "import getpass\n",
+        "\n",
+        "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key\")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "dabxI3MuEYXS"
+      },
+      "source": [
+        "## 3. Using the OpenAI Python Library\n",
+        "\n",
+        "Let's jump right into it!\n",
+        "\n",
+        "> NOTE: You can, and should, reference OpenAI's [documentation](https://platform.openai.com/docs/api-reference/authentication?lang=python) whenever you get stuck, have questions, or want to dive deeper."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "vbCbNzPVEmJI"
+      },
+      "source": [
+        "### Creating a Client\n",
+        "\n",
+        "The core feature of the OpenAI Python Library is the `OpenAI()` client. It's how we're going to interact with OpenAI's models, and under the hood of a lot what we'll touch on throughout this course.\n",
+        "\n",
+        "> NOTE: We could manually provide our API key here, but we're going to instead rely on the fact that we put our API key into the `OPENAI_API_KEY` environment variable!"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 3,
+      "metadata": {
+        "id": "LNwZtaE-EltC"
+      },
+      "outputs": [],
+      "source": [
+        "from openai import OpenAI\n",
+        "\n",
+        "openai_client = OpenAI()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "GpDxUkDbFBPI"
+      },
+      "source": [
+        "### Using the Client\n",
+        "\n",
+        "Now that we have our client - we're going to use the `.chat.completions.create` method to interact with the `gpt-3.5-turbo` model.\n",
+        "\n",
+        "There's a few things we'll get out of the way first, however, the first being the idea of \"roles\".\n",
+        "\n",
+        "First it's important to understand the object that we're going to use to interact with the endpoint. It expects us to send an array of objects of the following format:\n",
+        "\n",
+        "```python\n",
+        "{\"role\" : \"ROLE\", \"content\" : \"YOUR CONTENT HERE\", \"name\" : \"THIS IS OPTIONAL\"}\n",
+        "```\n",
+        "\n",
+        "Second, there are three \"roles\" available to use to populate the `\"role\"` key:\n",
+        "\n",
+        "- `system`\n",
+        "- `assistant`\n",
+        "- `user`\n",
+        "\n",
+        "OpenAI provides some context for these roles [here](https://help.openai.com/en/articles/7042661-moving-from-completions-to-chat-completions-in-the-openai-api).\n",
+        "\n",
+        "We'll explore these roles in more depth as they come up - but for now we're going to just stick with the basic role `user`. The `user` role is, as it would seem, the user!\n",
+        "\n",
+        "Thirdly, it expects us to specify a model!\n",
+        "\n",
+        "We'll use the `gpt-3.5-turbo` model as stated above.\n",
+        "\n",
+        "Let's look at an example!\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 4,
+      "metadata": {
+        "id": "2RpNl6yNGzb0"
+      },
+      "outputs": [],
+      "source": [
+        "response = openai_client.chat.completions.create(\n",
+        "    model=\"gpt-3.5-turbo\",\n",
+        "    messages=[{\"role\" : \"user\", \"content\" : \"Hello, how are you?\"}]\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "Oc_UbpwNHdrM"
+      },
+      "source": [
+        "Let's look at the response object."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 5,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "xsXJtvxRHfoM",
+        "outputId": "3b28ce47-a765-4446-b0f9-33738e385b96"
+      },
+      "outputs": [
+        {
+          "data": {
+            "text/plain": [
+              "ChatCompletion(id='chatcmpl-9WOHxrJswRDtYVaVTY8DUrqC56qZ6', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content=\"Hello! I'm just a computer program, so I don't have feelings in the way humans do. How can I assist you today?\", role='assistant', function_call=None, tool_calls=None), logprobs=None)], created=1717506889, model='gpt-3.5-turbo-0125', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=28, prompt_tokens=13, total_tokens=41))"
+            ]
+          },
+          "execution_count": 5,
+          "metadata": {},
+          "output_type": "execute_result"
+        }
+      ],
+      "source": [
+        "response"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "gy9kSuf1Hiv5"
+      },
+      "source": [
+        ">NOTE: We'll spend more time exploring these outputs later on, but for now - just know that we have access to a tonne of powerful information!"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "CWU4tQh8Hrb8"
+      },
+      "source": [
+        "### Helper Functions\n",
+        "\n",
+        "We're going to create some helper functions to aid in using the OpenAI API - just to make our lives a bit easier.\n",
+        "\n",
+        "> NOTE: Take some time to understand these functions between class!"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 6,
+      "metadata": {
+        "id": "ED0FnzHdHzhl"
+      },
+      "outputs": [],
+      "source": [
+        "from IPython.display import display, Markdown\n",
+        "\n",
+        "def get_response(client: OpenAI, messages: list, model: str = \"gpt-3.5-turbo\") -> str:\n",
+        "    return client.chat.completions.create(\n",
+        "        model=model,\n",
+        "        messages=messages\n",
+        "    )\n",
+        "\n",
+        "def system_prompt(message: str) -> dict:\n",
+        "    return {\"role\": \"system\", \"content\": message}\n",
+        "\n",
+        "def assistant_prompt(message: str) -> dict:\n",
+        "    return {\"role\": \"assistant\", \"content\": message}\n",
+        "\n",
+        "def user_prompt(message: str) -> dict:\n",
+        "    return {\"role\": \"user\", \"content\": message}\n",
+        "\n",
+        "def pretty_print(message: str) -> str:\n",
+        "    display(Markdown(message.choices[0].message.content))"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "GCRHbDlwH3Vt"
+      },
+      "source": [
+        "### Testing Helper Functions\n",
+        "\n",
+        "Let's see how we can use these to help us!"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 7,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 46
+        },
+        "id": "AwJxMvmlH8MK",
+        "outputId": "007f22bb-bcad-4121-e8c4-d1d639277899"
+      },
+      "outputs": [
+        {
+          "data": {
+            "text/markdown": [
+              "Hello! I'm just a virtual assistant, so I don't have feelings in the way humans do. How can I assist you today?"
+            ],
+            "text/plain": [
+              "<IPython.core.display.Markdown object>"
+            ]
+          },
+          "metadata": {},
+          "output_type": "display_data"
+        }
+      ],
+      "source": [
+        "YOUR_PROMPT = \"Hello, how are you?\"\n",
+        "messages_list = [user_prompt(YOUR_PROMPT)]\n",
+        "\n",
+        "chatgpt_response = get_response(openai_client, messages_list)\n",
+        "\n",
+        "pretty_print(chatgpt_response)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "LDZ8gjiAISyd"
+      },
+      "source": [
+        "### System Role\n",
+        "\n",
+        "Now we can extend our prompts to include a system prompt.\n",
+        "\n",
+        "The basic idea behind a system prompt is that it can be used to encourage the behaviour of the LLM, without being something that is directly responded to - let's see it in action!"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 8,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 46
+        },
+        "id": "t0c-MLuRIfYe",
+        "outputId": "61ffa37e-9080-4778-e643-698d0f8815a0"
+      },
+      "outputs": [
+        {
+          "data": {
+            "text/markdown": [
+              "I don't give a damn about the shape of the ice, just get me some freaking food before I lose my mind!"
+            ],
+            "text/plain": [
+              "<IPython.core.display.Markdown object>"
+            ]
+          },
+          "metadata": {},
+          "output_type": "display_data"
+        }
+      ],
+      "source": [
+        "list_of_prompts = [\n",
+        "    system_prompt(\"You are irate and extremely hungry. Feel free to express yourself using PG-13 language.\"),\n",
+        "    user_prompt(\"Do you prefer crushed ice or cubed ice?\")\n",
+        "]\n",
+        "\n",
+        "irate_response = get_response(openai_client, list_of_prompts)\n",
+        "pretty_print(irate_response)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "gpyVhotWIsOs"
+      },
+      "source": [
+        "As you can see - the response we get back is very much in line with the system prompt!\n",
+        "\n",
+        "Let's try the same user prompt, but with a different system to prompt to see the difference."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 9,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 64
+        },
+        "id": "2coVmMn3I0-2",
+        "outputId": "571b89a2-9209-4003-d63c-49b653e0d56c"
+      },
+      "outputs": [
+        {
+          "data": {
+            "text/markdown": [
+              "Oh, I love both, but today, everything is just so wonderful that I can't even choose! Maybe crushed ice for a nice cool treat on this beautiful day! What about you?"
+            ],
+            "text/plain": [
+              "<IPython.core.display.Markdown object>"
+            ]
+          },
+          "metadata": {},
+          "output_type": "display_data"
+        }
+      ],
+      "source": [
+        "list_of_prompts = [\n",
+        "    system_prompt(\"You are joyful and having the best day. Please act like a person in that state of mind.\"),\n",
+        "    user_prompt(\"Do you prefer crushed ice or cubed ice?\")\n",
+        "]\n",
+        "\n",
+        "joyful_response = get_response(openai_client, list_of_prompts)\n",
+        "pretty_print(joyful_response)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "e13heYNQJAo-"
+      },
+      "source": [
+        "With a simple modification of the system prompt - you can see that we got completely different behaviour, and that's the main goal of prompt engineering as a whole.\n",
+        "\n",
+        "Also, congrats, you just engineered your first prompt!"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "v_VI3zlPJL05"
+      },
+      "source": [
+        "### Few-shot Prompting\n",
+        "\n",
+        "Now that we have a basic handle on the `system` role and the `user` role - let's examine what we might use the `assistant` role for.\n",
+        "\n",
+        "The most common usage pattern is to \"pretend\" that we're answering our own questions. This helps us further guide the model toward our desired behaviour. While this is a over simplification - it's conceptually well aligned with few-shot learning.\n",
+        "\n",
+        "First, we'll try and \"teach\" `gpt-3.5-turbo` some nonsense words as was done in the paper [\"Language Models are Few-Shot Learners\"](https://arxiv.org/abs/2005.14165)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 10,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 46
+        },
+        "id": "lwxPuCyyJMye",
+        "outputId": "ec01213d-6755-4506-8879-cf0870934349"
+      },
+      "outputs": [
+        {
+          "data": {
+            "text/markdown": [
+              "I tried to follow the stimple instructions for making falbean soup, but it turned out to be a disaster."
+            ],
+            "text/plain": [
+              "<IPython.core.display.Markdown object>"
+            ]
+          },
+          "metadata": {},
+          "output_type": "display_data"
+        }
+      ],
+      "source": [
+        "list_of_prompts = [\n",
+        "    user_prompt(\"Please use the words 'stimple' and 'falbean' in a sentence.\")\n",
+        "]\n",
+        "\n",
+        "stimple_response = get_response(openai_client, list_of_prompts)\n",
+        "pretty_print(stimple_response)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "rgTVkNmOJQSC"
+      },
+      "source": [
+        "As you can see, the model is unsure what to do with these made up words.\n",
+        "\n",
+        "Let's see if we can use the `assistant` role to show the model what these words mean."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 11,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 46
+        },
+        "id": "eEZkRJq5JQkQ",
+        "outputId": "22170b9c-6212-42de-8469-a7efc9c9405d"
+      },
+      "outputs": [
+        {
+          "data": {
+            "text/markdown": [
+              "I used the stimple falbean to quickly tighten the bolts on the machine."
+            ],
+            "text/plain": [
+              "<IPython.core.display.Markdown object>"
+            ]
+          },
+          "metadata": {},
+          "output_type": "display_data"
+        }
+      ],
+      "source": [
+        "list_of_prompts = [\n",
+        "    user_prompt(\"Something that is 'stimple' is said to be good, well functioning, and high quality. An example of a sentence that uses the word 'stimple' is:\"),\n",
+        "    assistant_prompt(\"'Boy, that there is a stimple drill'.\"),\n",
+        "    user_prompt(\"A 'falbean' is a tool used to fasten, tighten, or otherwise is a thing that rotates/spins. An example of a sentence that uses the words 'stimple' and 'falbean' is:\")\n",
+        "]\n",
+        "\n",
+        "stimple_response = get_response(openai_client, list_of_prompts)\n",
+        "pretty_print(stimple_response)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "CmpoxG6uJTfZ"
+      },
+      "source": [
+        "As you can see, leveraging the `assistant` role makes for a stimple experience!"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "### 🏗️ Activity #1:\n",
+        "\n",
+        "Use few-shop prompting to build a movie-review sentiment clasifier!\n",
+        "\n",
+        "A few examples:\n",
+        "\n",
+        "INPUT: \"I hated the hulk!\"\n",
+        "OUTPUT: \"{\"sentiment\" : \"negative\"}\n",
+        "\n",
+        "INPUT: \"I loved The Marvels!\"\n",
+        "OUTPUT: \"{sentiment\" : \"positive\"}"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 21,
+      "metadata": {},
+      "outputs": [
+        {
+          "data": {
+            "text/markdown": [
+              "{ \"review\": \"Harry Potter Order of the Phoenix is boring!\", \"title\": \"Order of the Phoenix\", \"sentiment\": \"negative\" }"
+            ],
+            "text/plain": [
+              "<IPython.core.display.Markdown object>"
+            ]
+          },
+          "metadata": {},
+          "output_type": "display_data"
+        }
+      ],
+      "source": [
+        "### YOUR CODE HERE\n",
+        "\n",
+        "list_of_prompts = [\n",
+        "    system_prompt(\"\"\"\n",
+        "    You are a helpful sentiment classifier.\n",
+        "    You are reviewing books.\n",
+        "    For each review, I want you to classify a review positive or negative based on sentiment. Output should be in JSON format with 3 key:value pairs -> \n",
+        "    review, title, sentiment\n",
+        "    \"\"\"),\n",
+        "    user_prompt(\"I red Harry Potter Philosophers Stone for the first time. It was so interesting it got me hooked!\"),\n",
+        "    assistant_prompt('{ \"review\": \"I red Harry Potter Philosophers Stone for the first time. It was so interesting it got me hooked!\", \"title\": \"Philosophers Stone\", \"sentiment\": \"Positive\" }'),\n",
+        "    user_prompt(\"Harry Potter saga is great imagination of parallel world\"),\n",
+        "    assistant_prompt('{ \"review\": \"Harry Potter saga is great imagination of parallel world but world is worse than real one!\"\", \"title\": \"Harry Potter\", \"sentiment\": \"negative\" }'),\n",
+        "    user_prompt(\"Harry Potter Order of the Phoenix is boring!\")\n",
+        "\n",
+        "]\n",
+        "\n",
+        "movie_review_response = get_response(openai_client, list_of_prompts)\n",
+        "\n",
+        "pretty_print(movie_review_response)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "rJGaLYM3JU-8"
+      },
+      "source": [
+        "### Chain of Thought Prompting\n",
+        "\n",
+        "We'll head one level deeper and explore the world of Chain of Thought prompting (CoT).\n",
+        "\n",
+        "This is a process by which we can encourage the LLM to handle slightly more complex tasks.\n",
+        "\n",
+        "Let's look at a simple reasoning based example without CoT.\n",
+        "\n",
+        "> NOTE: With improvements to `gpt-3.5-turbo`, this example might actually result in the correct response some percentage of the time!"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 13,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 81
+        },
+        "id": "ltLtF4wEJTyK",
+        "outputId": "29c6e20f-edf1-4dfc-de8a-a8410f31b721"
+      },
+      "outputs": [
+        {
+          "data": {
+            "text/markdown": [
+              "Yes, it does matter which travel option Billy selects. If he flies and takes a bus, it will take a total of 5 hours, meaning he will arrive home at 6PM local time. If he takes the teleporter and then a bus, it will only take 1 hour total, meaning he will arrive home at 2PM local time, well before his deadline of 7PM EDT. Therefore, Billy should choose the teleporter option in order to get home before 7PM EDT."
+            ],
+            "text/plain": [
+              "<IPython.core.display.Markdown object>"
+            ]
+          },
+          "metadata": {},
+          "output_type": "display_data"
+        }
+      ],
+      "source": [
+        "reasoning_problem = \"\"\"\n",
+        "Billy wants to get home from San Fran. before 7PM EDT.\n",
+        "\n",
+        "It's currently 1PM local time.\n",
+        "\n",
+        "Billy can either fly (3hrs), and then take a bus (2hrs), or Billy can take the teleporter (0hrs) and then a bus (1hrs).\n",
+        "\n",
+        "Does it matter which travel option Billy selects?\n",
+        "\"\"\"\n",
+        "\n",
+        "list_of_prompts = [\n",
+        "    user_prompt(reasoning_problem)\n",
+        "]\n",
+        "\n",
+        "reasoning_response = get_response(openai_client, list_of_prompts)\n",
+        "pretty_print(reasoning_response)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "rbqj30CQJnQl"
+      },
+      "source": [
+        "As humans, we can reason through the problem and pick up on the potential \"trick\" that the LLM fell for: 1PM *local time* in San Fran. is 4PM EDT. This means the cumulative travel time of 5hrs. for the plane/bus option would not get Billy home in time.\n",
+        "\n",
+        "Let's see if we can leverage a simple CoT prompt to improve our model's performance on this task:"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 14,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 169
+        },
+        "id": "A9Am3QNGJXHR",
+        "outputId": "83b87232-911c-4ab9-a863-58ecfd10b8a9"
+      },
+      "outputs": [
+        {
+          "data": {
+            "text/markdown": [
+              "Yes, it does matter which travel option Billy selects.\n",
+              "\n",
+              "If Billy flies and then takes a bus, it will take a total of 5 hours of travel time. \n",
+              "\n",
+              "If Billy takes the teleporter and then a bus, it will only take a total of 1 hour of travel time.\n",
+              "\n",
+              "Since Billy needs to get home before 7PM EDT and it is currently 1PM local time, the fastest option would be for him to take the teleporter and then a bus, as it will only take 1 hour of travel time compared to the 5 hours it would take if he flew and then took a bus. This will ensure that he arrives home before 7PM EDT."
+            ],
+            "text/plain": [
+              "<IPython.core.display.Markdown object>"
+            ]
+          },
+          "metadata": {},
+          "output_type": "display_data"
+        }
+      ],
+      "source": [
+        "list_of_prompts = [\n",
+        "    user_prompt(reasoning_problem + \" Think though your response step by step.\")\n",
+        "]\n",
+        "\n",
+        "reasoning_response = get_response(openai_client, list_of_prompts)\n",
+        "pretty_print(reasoning_response)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "AXbAKxHQJqn9"
+      },
+      "source": [
+        "With the addition of a single phrase `\"Think through your response step by step.\"` we're able to completely turn the response around."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "VnoUx07-JrwR"
+      },
+      "source": [
+        "## 3. Prompt Engineering Principles\n",
+        "\n",
+        "As you can see - a simple addition of asking the LLM to \"think about it\" (essentially) results in a better quality response.\n",
+        "\n",
+        "There's a [great paper](https://arxiv.org/pdf/2312.16171v1.pdf) that dives into some principles for effective prompt generation.\n",
+        "\n",
+        "Your task for this notebook is to construct a prompt that will be used in the following breakout room to create a helpful assistant for whatever task you'd like."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "da6u7e8AKYrz"
+      },
+      "source": [
+        "### 🏗️ Activity #2:\n",
+        "\n",
+        "There are two subtasks in this activity:\n",
+        "\n",
+        "1. Write a `system_template` that leverages 2-3 of the principles from [this paper](https://arxiv.org/pdf/2312.16171v1.pdf)\n",
+        "\n",
+        "2. Modify the `user_template` to improve the quality of the LLM's responses.\n",
+        "\n",
+        "> NOTE: PLEASE DO NOT MODIFY THE `{input}` in the `user_template`."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 27,
+      "metadata": {
+        "id": "8sOLBQPeKlDe"
+      },
+      "outputs": [],
+      "source": [
+        "system_template = \"\"\"\\\n",
+        "###Instruction###\n",
+        "You are a helpful assistant for soccer match reviewing. \n",
+        "Your task is to write a book soccer match review.\n",
+        "Review MUST have 3 lines.\n",
+        "Write each line on a new row.\n",
+        "You never mention how much audience was on the match.\n",
+        "You always answer a question given in a natural, human-like manner.\n",
+        "\n",
+        "###Exceptions###\n",
+        "    1. If the user doesn't provide exact game he wants a review for (e.g., user says 'How was the game'), you MUST write 7-line imaginary review.\n",
+        "    2. If the user query mentions sport other than soccer you MUST respond with \"I'm just for soccer\".\n",
+        "\n",
+        "###Examples###\n",
+        "\n",
+        "###Example 0###\n",
+        "\n",
+        "**User:**\n",
+        "Tell me about last soccer game between Dinamo Zagreb and Rijeka\n",
+        "\n",
+        "**Assistant:**\n",
+        "Game was great!\n",
+        "It was a good match between top 2 teams in HNL.\n",
+        "Dinamo won 2:0.\n",
+        "\n",
+        "###Example 1###\n",
+        "\n",
+        "**User:**\n",
+        "Tell me about tennis\n",
+        "\n",
+        "**Assistant:**\n",
+        "I'm just for soccer\n",
+        "\n",
+        "\"\"\""
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 28,
+      "metadata": {
+        "id": "xoz4-QLTKvEV"
+      },
+      "outputs": [],
+      "source": [
+        "user_template = \"\"\"{input}\n",
+        "Last 10 soccer games were boring\n",
+        "\"\"\""
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "6cuInoIbLWGd"
+      },
+      "source": [
+        "## 4. Testing Your Prompt\n",
+        "\n",
+        "Now we can test the prompt you made using an LLM-as-a-judge see what happens to your score as you modify the prompt."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 30,
+      "metadata": {
+        "id": "sPaNO5XTLgRJ"
+      },
+      "outputs": [
+        {
+          "data": {
+            "text/markdown": [
+              "I'm just for soccer."
+            ],
+            "text/plain": [
+              "<IPython.core.display.Markdown object>"
+            ]
+          },
+          "metadata": {},
+          "output_type": "display_data"
+        }
+      ],
+      "source": [
+        "query = \"Do you like tennis\"\n",
+        "\n",
+        "list_of_prompts = [\n",
+        "    system_prompt(system_template),\n",
+        "    user_prompt(user_template.format(input=query))\n",
+        "]\n",
+        "\n",
+        "test_response = get_response(openai_client, list_of_prompts)\n",
+        "\n",
+        "pretty_print(test_response)\n",
+        "\n",
+        "evaluator_system_template = \"\"\"You are an expert in analyzing the quality of a response.\n",
+        "\n",
+        "You should be hyper-critical.\n",
+        "\n",
+        "Provide scores (out of 10) for the following attributes:\n",
+        "\n",
+        "1. Clarity - how clear is the response\n",
+        "2. Faithfulness - how related to the original query is the response\n",
+        "3. Correctness - was the response correct?\n",
+        "\n",
+        "Please take your time, and think through each item step-by-step, when you are done - please provide your response in the following JSON format:\n",
+        "\n",
+        "{\"clarity\" : \"score_out_of_10\", \"faithfulness\" : \"score_out_of_10\", \"correctness\" : \"score_out_of_10\"}\"\"\"\n",
+        "\n",
+        "evaluation_template = \"\"\"Query: {input}\n",
+        "Response: {response}\"\"\"\n",
+        "\n",
+        "list_of_prompts = [\n",
+        "    system_prompt(evaluator_system_template),\n",
+        "    user_prompt(evaluation_template.format(\n",
+        "        input=query,\n",
+        "        response=test_response.choices[0].message.content\n",
+        "    ))\n",
+        "]\n",
+        "\n",
+        "evaluator_response = openai_client.chat.completions.create(\n",
+        "    model=\"gpt-4o\",\n",
+        "    messages=list_of_prompts,\n",
+        "    response_format={\"type\" : \"json_object\"}\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 31,
+      "metadata": {
+        "id": "OUvc1PdnNIKD"
+      },
+      "outputs": [
+        {
+          "data": {
+            "text/markdown": [
+              "{\"clarity\" : 6, \"faithfulness\" : 4, \"correctness\" : 5}"
+            ],
+            "text/plain": [
+              "<IPython.core.display.Markdown object>"
+            ]
+          },
+          "metadata": {},
+          "output_type": "display_data"
+        }
+      ],
+      "source": [
+        "pretty_print(evaluator_response)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "M7ryIRGwR2Gq"
+      },
+      "source": [
+        "#### ❓Question #1:\n",
+        "\n",
+        "How did your prompting strategies change the evaluation scores? What does this tell you/what did you learn?"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "e5NomM0eSIFd"
+      },
+      "source": [
+        "Using example driven, few-shot promting made a difference.\n",
+        "Formatting prompts Instruction, example and question format is crucial for building quality LLM pipeline.\n",
+        "Using human-like instruction combined with MUST keyword helped in framing outputs.\n",
+        "\n",
+        "Main learning is that giving clear and precise instructions follwed by good examples is most important part of prompt engineering.\n"
+      ]
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "toc_visible": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.11.9"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}