UI_design_oral_exam_baseline_functionality.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "V100",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/vanderbilt-data-science/lo-achievement/blob/124-implement-baseline-functionality-for-oral-exam-module/UI_design_oral_exam_baseline_functionality.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Project IO Achievement - UI Design (Oral Exam)"
      ],
      "metadata": {
        "id": "PIbogPXyM0wr"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Problem Definition\n",
        "\n",
        "The v1 functionality for the Oral Exam module requires the following:\n",
        "\n",
        "1. Upload or generation of questions: either the user should upload a set of questions or we should allow the model to generate the questions. The user should pick or it should be inherent if there is no upload of questions. Note that we must also allow for context to be uploaded (vector store, vector store link, specific documents)\n",
        "2. The model should prompt the user with a question and pause.\n",
        "The user should respond by audio.\n",
        "3. This should continue on until some final point where the exam is over.\n",
        "\n",
        "Then:\n",
        "\n",
        "1. We should use Whisper to do the transcription, and\n",
        "2. Send the transcription, questions, and context for GPT4 for evaluation\n",
        "Return the evaluation.\n",
        "3. This will primarily be work on a user interface."
      ],
      "metadata": {
        "id": "x_Vp8SiKM4p1"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Libraries\n",
        "\n",
        "This section will install and import some important libraries such as Langchain, openai, Gradio, and so on"
      ],
      "metadata": {
        "id": "o_60X8H3NEne"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# install libraries here\n",
        "# -q flag for \"quiet\" install\n",
        "%%capture\n",
        "!pip install -q langchain\n",
        "!pip install -q openai\n",
        "!pip install -q gradio\n",
        "!pip install -q transformers\n",
        "!pip install -q datasets\n",
        "!pip install -q huggingsound\n",
        "!pip install -q torchaudio\n",
        "!pip install -q git+https://github.com/openai/whisper.git\n",
        "!pip install -q docx\n",
        "!pip install -q PyPDF2\n",
        "!pip install python-docx"
      ],
      "metadata": {
        "id": "pxcqXgg2aAN7"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "pEjM1tLsMZBq"
      },
      "outputs": [],
      "source": [
        "# import libraries here\n",
        "from langchain.llms import OpenAI\n",
        "from langchain.prompts import PromptTemplate\n",
        "from langchain.document_loaders import TextLoader\n",
        "from langchain.indexes import VectorstoreIndexCreator\n",
        "from langchain import ConversationChain, LLMChain, PromptTemplate\n",
        "from langchain.chat_models import ChatOpenAI\n",
        "from langchain.memory import ConversationBufferWindowMemory\n",
        "from langchain.prompts import ChatPromptTemplate\n",
        "from langchain.text_splitter import CharacterTextSplitter\n",
        "from langchain.embeddings import OpenAIEmbeddings\n",
        "from langchain.schema import SystemMessage, HumanMessage, AIMessage\n",
        "import openai\n",
        "import os\n",
        "from getpass import getpass\n",
        "from IPython.display import display, Javascript, HTML\n",
        "from google.colab.output import eval_js\n",
        "from base64 import b64decode\n",
        "import ipywidgets as widgets\n",
        "from IPython.display import clear_output\n",
        "import time\n",
        "import requests\n",
        "from transformers import WhisperProcessor, WhisperForConditionalGeneration\n",
        "from datasets import load_dataset\n",
        "# from torchaudio.transforms import Resample\n",
        "import whisper\n",
        "from huggingsound import SpeechRecognitionModel\n",
        "import numpy as np\n",
        "import torch\n",
        "import librosa\n",
        "from datasets import load_dataset\n",
        "from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor\n",
        "from jiwer import wer\n",
        "import pandas as pd\n",
        "from IPython.display import display, HTML\n",
        "import gradio as gr\n",
        "from transformers import pipeline\n",
        "from docx import Document\n",
        "import PyPDF2\n",
        "from pydub import AudioSegment\n",
        "import tempfile\n",
        "import os\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## API Keys\n",
        "\n",
        "Use these cells to load the API keys required for this notebook. The below code cell uses the `getpass` library."
      ],
      "metadata": {
        "id": "03KLZGI_a5W5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "openai_api_key = getpass()\n",
        "os.environ[\"OPENAI_API_KEY\"] = openai_api_key\n",
        "openai.api_key = openai_api_key"
      ],
      "metadata": {
        "id": "5smcWj4DbFgy",
        "outputId": "9a73707b-1a6a-4253-b7d8-181a82b1040f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 3,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "··········\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Prompt Design\n",
        "\n",
        "To be added"
      ],
      "metadata": {
        "id": "pMo9x8u4AEV1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "chat = ChatOpenAI(temperature=0.0, model_name='gpt-3.5-turbo-16k')\n",
        "chat"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UgnCZRMhADvo",
        "outputId": "1bd6b84d-3ea8-49ba-8156-701f4155d69c"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "ChatOpenAI(cache=None, verbose=False, callbacks=None, callback_manager=None, tags=None, metadata=None, client=<class 'openai.api_resources.chat_completion.ChatCompletion'>, model_name='gpt-3.5-turbo-16k', temperature=0.0, model_kwargs={}, openai_api_key='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', openai_api_base='', openai_organization='', openai_proxy='', request_timeout=None, max_retries=6, streaming=False, n=1, max_tokens=None, tiktoken_model_name=None)"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# This is what I used to test the function 'generate_questions'\n",
        "template_string2 = \"\"\"\n",
        "You are teacher, and you will be given a {context} that is related to the presentation topic.\n",
        "\n",
        "Please generate a questions based on the context above and transcript that student created . \\\n",
        "\n",
        "The audio file generated by student is shown below: {transcribed_text}. \\\n",
        "\"\"\""
      ],
      "metadata": {
        "id": "WmysQZAhKBli"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "prompt_template1 = ChatPromptTemplate.from_template(template_string2)"
      ],
      "metadata": {
        "id": "oij6W5rwAaGb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# prompt_template.messages[0].prompt\n",
        "prompt_template1.messages[0].prompt.input_variables"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C1YRmL46AaJA",
        "outputId": "e850524a-3831-4113-c796-5a0ec8584569"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['context', 'transcribed_text']"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# This template is used for testing the function 'ai_evaluate'\n",
        "# Detailed evaluation metrics are to be added\n",
        "template_string3 = \"\"\"\n",
        "You are teacher, and you will be given a context that is related to the presentation topic. \\\n",
        "Now, given {context}, evaluate the answer based on the accuracy\n",
        "\n",
        "The main answer generated by student is shown below: {transcribed_text}. \\\n",
        "The questions are shown below: {questions}. \\\n",
        "The questions answered by student is shown below: {transcribed_qa}. \\\n",
        "\"\"\"\n",
        "prompt_template2 = ChatPromptTemplate.from_template(template_string3)\n",
        "prompt_template2.messages[0].prompt.input_variables\n"
      ],
      "metadata": {
        "id": "141Cxa2MT-l7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0374d45d-a7f6-41e7-aed0-44c61681de21"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['context', 'questions', 'transcribed_qa', 'transcribed_text']"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Integrate Prompts from LO project"
      ],
      "metadata": {
        "id": "MJCHl1T2TPWC"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Creating a Chain for Short Answer Generation"
      ],
      "metadata": {
        "id": "IPTyUOl-WdiL"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this example, the context would include the poem \"The Road Not Taken\" by Robert Frost"
      ],
      "metadata": {
        "id": "203qBjZvmFK1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# This is what I used to test the function 'generate_questions_v2'\n",
        "template_string = \"\"\"\n",
        "You are a world-class tutor helping students to perform better on oral and written exams though interactive experiences.\"\n",
        "\n",
        "The following text should be used as the basis for the instructions which follow: {context} \\\n",
        "\n",
        "The following is the guideline for generating the questiion: {pre_prompt}\n",
        "\"\"\""
      ],
      "metadata": {
        "id": "w1AjHwIoVnvw"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "prompt_template = ChatPromptTemplate.from_template(template_string)\n",
        "prompt_template.messages[0].prompt.input_variables"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "39-lm5I-Wlep",
        "outputId": "44b39930-5258-484b-8c7d-c36ff4b5dc1a"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['context', 'pre_prompt']"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Creating a Chain for AI Evaluation"
      ],
      "metadata": {
        "id": "-Pfxkcdxh9nZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "template_evaluation = \"\"\"\n",
        "Given the follwing {context} and the {transcript}, evaluate whether or not the student answered correctly on the {question}.\n",
        "\"\"\""
      ],
      "metadata": {
        "id": "u6pH1x-gWnFF"
      },
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# @title\n",
        "prompt_template2 = ChatPromptTemplate.from_template(template_evaluation)\n",
        "prompt_template2.messages[0].prompt.input_variables"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YPO_IE5ThC6W",
        "outputId": "5361929c-cf8c-483d-901a-ed14a0db89fa"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['context', 'question', 'transcript']"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## UI Design\n",
        "\n",
        "https://colab.research.google.com/github/petewarden/openai-whisper-webapp/blob/main/OpenAI_Whisper_ASR_Demo.ipynb"
      ],
      "metadata": {
        "id": "M6IzVTjz5cex"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Functions"
      ],
      "metadata": {
        "id": "l4o8R5eUE1n8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def embed_key(openai_api_key):\n",
        "  os.environ[\"OPENAI_API_KEY\"] = openai_api_key\n",
        "\n",
        "def transcribe(audio_file_path):\n",
        "    with open(audio_file_path, \"rb\") as audio_file:\n",
        "        # Call OpenAI's Whisper model for transcription\n",
        "        transcript = openai.Audio.transcribe(\"whisper-1\", audio_file)\n",
        "        transcribed_text = transcript[\"text\"]\n",
        "        return transcribed_text\n",
        "\n",
        "def translate(text):\n",
        "    # Create a prompt template (This will be changed later to fit the actual task)\n",
        "    # Here translation will be a filler task of GPT\n",
        "    test_input1 = prompt_template.format_messages(\n",
        "              expertise='Language Translation',\n",
        "              language='Japanese',\n",
        "              style='romantic',\n",
        "              transcribed_text=text)\n",
        "\n",
        "    response = chat.predict_messages(test_input1)\n",
        "    return response.content\n",
        "\n",
        "def process_file(files):\n",
        "  for file in files:\n",
        "    try:\n",
        "        extension = file.name.split('.')[-1].lower()\n",
        "        if extension == 'docx':\n",
        "            doc = Document(file.name)\n",
        "            full_text = []\n",
        "            for paragraph in doc.paragraphs:\n",
        "                full_text.append(paragraph.text)\n",
        "            return '\\n'.join(full_text)\n",
        "\n",
        "        elif extension == 'pdf':\n",
        "            pdf_file = open(file.name, 'rb')\n",
        "            reader = PyPDF2.PdfReader(pdf_file)\n",
        "            num_pages = len(reader.pages)\n",
        "            full_text = []\n",
        "            for page in range(num_pages):\n",
        "                page_obj = reader.pages[page]\n",
        "                full_text.append(page_obj.extract_text())\n",
        "            pdf_file.close()\n",
        "            return '\\n'.join(full_text)\n",
        "\n",
        "        elif extension == 'txt':\n",
        "            with open(file.name, 'r') as txt_file:\n",
        "                full_text = txt_file.read()\n",
        "            return full_text\n",
        "\n",
        "        else:\n",
        "            return \"Unsupported file type\"\n",
        "    except FileNotFoundError:\n",
        "        return \"File not found\"\n",
        "    except PermissionError:\n",
        "        return \"Permission denied\"\n",
        "\n",
        "def generate_questions(context, transcript):\n",
        "    text = process_file(context)\n",
        "    test_input1 = prompt_template1.format_messages(\n",
        "                      context = text,\n",
        "                      transcribed_text = transcript)\n",
        "\n",
        "    response = chat(test_input1)\n",
        "    return response.content\n",
        "\n",
        "def generate_questions_v2(text, prompt):\n",
        "    #text = process_file(file)\n",
        "    test_input1 = prompt_template.format_messages(\n",
        "                      context = text,\n",
        "                      pre_prompt = prompt)\n",
        "\n",
        "    response = chat(test_input1)\n",
        "    return response\n",
        "\n",
        "# def ai_evaluate(context, audio_main, audio_qa, questions):\n",
        "#     test_input1 = prompt_template2.format_messages(\n",
        "#                       context = context,\n",
        "#                       transcribed_text = audio_main,\n",
        "#                       transcribed_qa = audio_qa,\n",
        "#                       questions = questions)\n",
        "\n",
        "#     response = chat(test_input1)\n",
        "#     return response.content\n",
        "\n",
        "def ai_evaluate_v2(text, audio_main, questions):\n",
        "    #audio = transcribe(audio_main)\n",
        "    test_input1 = prompt_template2.format_messages(\n",
        "                      context = text,\n",
        "                      transcript = audio_main,\n",
        "                      question = questions\n",
        "                      )\n",
        "\n",
        "    response = chat(test_input1)\n",
        "    return response.content\n",
        "\n",
        "def upload_file(files):\n",
        "    file_paths = [file.name for file in files]\n",
        "    return file_paths"
      ],
      "metadata": {
        "id": "ABN0X9xQHeii"
      },
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Test process_file"
      ],
      "metadata": {
        "id": "a3WUL_hFyMkr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import files\n",
        "def upload_syllabi():\n",
        "  uploaded = files.upload()\n",
        "  for name, data in uploaded.items():\n",
        "    with open(name, 'wb') as f:\n",
        "      f.write(data)\n",
        "    print('saved file', name)\n",
        "upload_syllabi()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 90
        },
        "id": "nih4FXX0Pl9U",
        "outputId": "ce48c70a-d52c-4267-f3fc-8b22404448d7"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-c72e8e8a-ac6f-48ab-9fb4-a84b1483268a\" name=\"files[]\" multiple disabled\n",
              "        style=\"border:none\" />\n",
              "     <output id=\"result-c72e8e8a-ac6f-48ab-9fb4-a84b1483268a\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script>// Copyright 2017 Google LLC\n",
              "//\n",
              "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
              "// you may not use this file except in compliance with the License.\n",
              "// You may obtain a copy of the License at\n",
              "//\n",
              "//      http://www.apache.org/licenses/LICENSE-2.0\n",
              "//\n",
              "// Unless required by applicable law or agreed to in writing, software\n",
              "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
              "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
              "// See the License for the specific language governing permissions and\n",
              "// limitations under the License.\n",
              "\n",
              "/**\n",
              " * @fileoverview Helpers for google.colab Python module.\n",
              " */\n",
              "(function(scope) {\n",
              "function span(text, styleAttributes = {}) {\n",
              "  const element = document.createElement('span');\n",
              "  element.textContent = text;\n",
              "  for (const key of Object.keys(styleAttributes)) {\n",
              "    element.style[key] = styleAttributes[key];\n",
              "  }\n",
              "  return element;\n",
              "}\n",
              "\n",
              "// Max number of bytes which will be uploaded at a time.\n",
              "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
              "\n",
              "function _uploadFiles(inputId, outputId) {\n",
              "  const steps = uploadFilesStep(inputId, outputId);\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  // Cache steps on the outputElement to make it available for the next call\n",
              "  // to uploadFilesContinue from Python.\n",
              "  outputElement.steps = steps;\n",
              "\n",
              "  return _uploadFilesContinue(outputId);\n",
              "}\n",
              "\n",
              "// This is roughly an async generator (not supported in the browser yet),\n",
              "// where there are multiple asynchronous steps and the Python side is going\n",
              "// to poll for completion of each step.\n",
              "// This uses a Promise to block the python side on completion of each step,\n",
              "// then passes the result of the previous step as the input to the next step.\n",
              "function _uploadFilesContinue(outputId) {\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  const steps = outputElement.steps;\n",
              "\n",
              "  const next = steps.next(outputElement.lastPromiseValue);\n",
              "  return Promise.resolve(next.value.promise).then((value) => {\n",
              "    // Cache the last promise value to make it available to the next\n",
              "    // step of the generator.\n",
              "    outputElement.lastPromiseValue = value;\n",
              "    return next.value.response;\n",
              "  });\n",
              "}\n",
              "\n",
              "/**\n",
              " * Generator function which is called between each async step of the upload\n",
              " * process.\n",
              " * @param {string} inputId Element ID of the input file picker element.\n",
              " * @param {string} outputId Element ID of the output display.\n",
              " * @return {!Iterable<!Object>} Iterable of next steps.\n",
              " */\n",
              "function* uploadFilesStep(inputId, outputId) {\n",
              "  const inputElement = document.getElementById(inputId);\n",
              "  inputElement.disabled = false;\n",
              "\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  outputElement.innerHTML = '';\n",
              "\n",
              "  const pickedPromise = new Promise((resolve) => {\n",
              "    inputElement.addEventListener('change', (e) => {\n",
              "      resolve(e.target.files);\n",
              "    });\n",
              "  });\n",
              "\n",
              "  const cancel = document.createElement('button');\n",
              "  inputElement.parentElement.appendChild(cancel);\n",
              "  cancel.textContent = 'Cancel upload';\n",
              "  const cancelPromise = new Promise((resolve) => {\n",
              "    cancel.onclick = () => {\n",
              "      resolve(null);\n",
              "    };\n",
              "  });\n",
              "\n",
              "  // Wait for the user to pick the files.\n",
              "  const files = yield {\n",
              "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
              "    response: {\n",
              "      action: 'starting',\n",
              "    }\n",
              "  };\n",
              "\n",
              "  cancel.remove();\n",
              "\n",
              "  // Disable the input element since further picks are not allowed.\n",
              "  inputElement.disabled = true;\n",
              "\n",
              "  if (!files) {\n",
              "    return {\n",
              "      response: {\n",
              "        action: 'complete',\n",
              "      }\n",
              "    };\n",
              "  }\n",
              "\n",
              "  for (const file of files) {\n",
              "    const li = document.createElement('li');\n",
              "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
              "    li.append(span(\n",
              "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
              "        `last modified: ${\n",
              "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
              "                                    'n/a'} - `));\n",
              "    const percent = span('0% done');\n",
              "    li.appendChild(percent);\n",
              "\n",
              "    outputElement.appendChild(li);\n",
              "\n",
              "    const fileDataPromise = new Promise((resolve) => {\n",
              "      const reader = new FileReader();\n",
              "      reader.onload = (e) => {\n",
              "        resolve(e.target.result);\n",
              "      };\n",
              "      reader.readAsArrayBuffer(file);\n",
              "    });\n",
              "    // Wait for the data to be ready.\n",
              "    let fileData = yield {\n",
              "      promise: fileDataPromise,\n",
              "      response: {\n",
              "        action: 'continue',\n",
              "      }\n",
              "    };\n",
              "\n",
              "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
              "    let position = 0;\n",
              "    do {\n",
              "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
              "      const chunk = new Uint8Array(fileData, position, length);\n",
              "      position += length;\n",
              "\n",
              "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
              "      yield {\n",
              "        response: {\n",
              "          action: 'append',\n",
              "          file: file.name,\n",
              "          data: base64,\n",
              "        },\n",
              "      };\n",
              "\n",
              "      let percentDone = fileData.byteLength === 0 ?\n",
              "          100 :\n",
              "          Math.round((position / fileData.byteLength) * 100);\n",
              "      percent.textContent = `${percentDone}% done`;\n",
              "\n",
              "    } while (position < fileData.byteLength);\n",
              "  }\n",
              "\n",
              "  // All done.\n",
              "  yield {\n",
              "    response: {\n",
              "      action: 'complete',\n",
              "    }\n",
              "  };\n",
              "}\n",
              "\n",
              "scope.google = scope.google || {};\n",
              "scope.google.colab = scope.google.colab || {};\n",
              "scope.google.colab._files = {\n",
              "  _uploadFiles,\n",
              "  _uploadFilesContinue,\n",
              "};\n",
              "})(self);\n",
              "</script> "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Saving instructor_note_2.docx to instructor_note_2.docx\n",
            "saved file instructor_note_2.docx\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Might need some way to make pdf file to load more readable\n",
        "# process_file('/content/instrutor_note.docx')\n",
        "# process_file('/content/Big Data & Economics.pdf')\n",
        "# process_file('/content/Big Data & Economics.pdf')"
      ],
      "metadata": {
        "id": "LJX1AKTMyVm8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Gradio Interface V1"
      ],
      "metadata": {
        "id": "c4s5o8baE6wN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# @title\n",
        "# with gr.Blocks() as demo:\n",
        "#   gr.Markdown(\"# Oral Exam App\")\n",
        "#   with gr.Box():\n",
        "#     gr.HTML(\"\"\"Embed your OpenAI API key below; if you haven't created one already, visit\n",
        "#       platform.openai.com/account/api-keys\n",
        "#     to sign up for an account and get your personal API key\"\"\",\n",
        "#             elem_classes=\"textbox_label\")\n",
        "#     input = gr.Textbox(show_label=False, type=\"password\", container=False,\n",
        "#                       placeholder=\"●●●●●●●●●●●●●●●●●\")\n",
        "#     input.change(fn=embed_key, inputs=input, outputs=None)\n",
        "\n",
        "#   with gr.Blocks():\n",
        "#       gr.Markdown(\"## Upload your audio file or start recording\")\n",
        "\n",
        "#       with gr.Row():\n",
        "\n",
        "\n",
        "#         with gr.Column():\n",
        "#           file_input = gr.Files(label=\"Load a mp3 file\",\n",
        "#                               file_types=['.mp3'], type=\"file\",\n",
        "#                               elem_classes=\"short-height\")\n",
        "#           record_inputs = gr.Audio(source=\"microphone\", type=\"filepath\")\n",
        "\n",
        "#         with gr.Column():\n",
        "#           outputs_transcribe=gr.Textbox(label=\"Transcription\")\n",
        "\n",
        "#           with gr.Row():\n",
        "#             btn1 = gr.Button(value=\"Transcribe recorded audio\")\n",
        "#             btn1.click(transcribe, inputs=record_inputs, outputs=outputs_transcribe)\n",
        "#             btn2 = gr.Button(value=\"Transcribe uploaded audio\")\n",
        "#             btn2.click(transcribe, inputs=file_input, outputs=outputs_transcribe)\n",
        "\n",
        "#           outputs_translate=gr.Textbox(label=\"Translation\")\n",
        "#           btn3 = gr.Button(value=\"Translate\")\n",
        "#           btn3.click(translate, inputs=outputs_transcribe, outputs=outputs_translate)\n",
        "\n",
        "#   demo.launch()\n"
      ],
      "metadata": {
        "id": "ZkfJXCGDFhdw",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### baseline functionality V1"
      ],
      "metadata": {
        "id": "AnkuosJ7Vw4z"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# @title\n",
        "with gr.Blocks() as demo:\n",
        "  gr.Markdown(\"# Oral Exam App\")\n",
        "  gr.Markdown(\"## OpenAI API key\")\n",
        "  with gr.Box():\n",
        "    gr.HTML(\"\"\"Embed your OpenAI API key below; if you haven't created one already, visit\n",
        "      platform.openai.com/account/api-keys\n",
        "    to sign up for an account and get your personal API key\"\"\",\n",
        "            elem_classes=\"textbox_label\")\n",
        "    input = gr.Textbox(show_label=False, type=\"password\", container=False,\n",
        "                      placeholder=\"●●●●●●●●●●●●●●●●●\")\n",
        "    input.change(fn=embed_key, inputs=input, outputs=None)\n",
        "\n",
        "  with gr.Blocks():\n",
        "  #########################\n",
        "  #########Context#########\n",
        "  #########################\n",
        "      with gr.Accordion(\"Context section\"):\n",
        "      ### Should also allow vector stores\n",
        "          gr.Markdown(\"## Please upload the context document(s) for Oral exam\")\n",
        "          context_input = gr.File(label=\"Click to upload context file\",\n",
        "                                  file_count=\"multiple\",\n",
        "                                  file_types=[\".txt\", \".docx\", \".pdf\"])\n",
        "          outputs_context=gr.Textbox(label=\"Context\")\n",
        "          context_input.change(fn=process_file, inputs=context_input, outputs=outputs_context)\n",
        "          # upload_button = gr.Button(value=\"Show context\")\n",
        "          # upload_button.click(process_file, context_input, outputs_context)\n",
        "\n",
        "  #########################\n",
        "  #######Main Audio########\n",
        "  #########################\n",
        "      with gr.Accordion(\"Main audio section\"):\n",
        "        gr.Markdown(\"## Upload your audio file or start recording\")\n",
        "        with gr.Column():\n",
        "          ## uploading files seem not working (don't know why)\n",
        "          with gr.Row():\n",
        "              file_input = gr.Audio(label=\"Upload Audio\", source=\"upload\", type=\"filepath\")\n",
        "              record_inputs = gr.Audio(label=\"Record Audio\", source=\"microphone\", type=\"filepath\")\n",
        "\n",
        "          gr.Markdown(\"## Transcribe the audio uploaded or recorded\")\n",
        "          outputs_transcribe=gr.Textbox(label=\"Transcription\")\n",
        "\n",
        "          file_input.change(fn=transcribe, inputs=file_input, outputs=outputs_transcribe)\n",
        "          record_inputs.change(fn=transcribe, inputs=record_inputs, outputs=outputs_transcribe)\n",
        "\n",
        "  #########################\n",
        "  ###Question Generation###\n",
        "  #########################\n",
        "      with gr.Accordion(\"Question section\"):\n",
        "          gr.Markdown(\"## Questions\")\n",
        "          with gr.Row():\n",
        "            with gr.Column():\n",
        "              outputs_qa=gr.Textbox(label=\"Generate questions\")\n",
        "              btn3 = gr.Button(value=\"Generate questions\")\n",
        "              btn3.click(generate_questions, inputs=[context_input, outputs_transcribe], outputs=outputs_qa)\n",
        "\n",
        "  ######################### Need additional work to include these questions when click button   #########################\n",
        "            with gr.Column():\n",
        "              submit_question=gr.Textbox(label=\"Use existing questions\")\n",
        "              btn4 = gr.Button(value=\"Use these questions\")\n",
        "              # btn4.click(use_this_question, inputs=outputs_transcribe, outputs=None)\n",
        "\n",
        "  #########################\n",
        "  #########Audio QA########\n",
        "  #########################\n",
        "      with gr.Accordion(\"Audio QA section\"):\n",
        "          gr.Markdown(\"## Question answering\")\n",
        "          ##### This may be iterative\n",
        "          with gr.Row():\n",
        "            file_input2 = gr.Audio(label=\"Upload Audio\", source=\"upload\", type=\"filepath\")\n",
        "            record_inputs2 = gr.Audio(label=\"Record Audio\", source=\"microphone\", type=\"filepath\")\n",
        "\n",
        "          gr.Markdown(\"## Transcribe the audio uploaded or recorded\")\n",
        "          outputs_transcribe2=gr.Textbox(label=\"Transcription\")\n",
        "          file_input2.change(fn=transcribe, inputs=file_input2, outputs=outputs_transcribe2)\n",
        "          record_inputs2.change(fn=transcribe, inputs=record_inputs2, outputs=outputs_transcribe2)\n",
        "\n",
        "  #########################\n",
        "  #######Evaluation########\n",
        "  #########################\n",
        "      with gr.Accordion(\"Evaluation section\"):\n",
        "          gr.Markdown(\"## Evaluation\")\n",
        "          with gr.Tab(\"General evalution\"):\n",
        "            evalution=gr.Textbox(label=\"AI Evaluation\")\n",
        "            btn5 = gr.Button(value=\"Evaluate\")\n",
        "            btn5.click(ai_evaluate, inputs=[context_input, record_inputs,record_inputs2, outputs_qa], outputs=evalution)\n",
        "          with gr.Tab(\"Quantitative evalution\"):\n",
        "            table_output = gr.Dataframe(label = \"Some kind of evaluation metrics?\")\n",
        "            btn6 = gr.Button(value=\"Evaluate\")\n",
        "            btn6.click(ai_evaluate, inputs=[context_input, record_inputs,record_inputs2, outputs_qa], outputs=table_output)\n",
        "\n",
        "  demo.launch()\n",
        "  # demo.launch(share=True)\n",
        "  # demo.launch(debug=True)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 616
        },
        "id": "EAPljDMYVy3u",
        "outputId": "1f347376-14e8-48ea-e531-295a4fefd6cd",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Colab notebook detected. To show errors in colab notebook, set debug=True in launch()\n",
            "Note: opening Chrome Inspector may crash demo inside Colab notebooks.\n",
            "\n",
            "To create a public link, set `share=True` in `launch()`.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ],
            "application/javascript": [
              "(async (port, path, width, height, cache, element) => {\n",
              "                        if (!google.colab.kernel.accessAllowed && !cache) {\n",
              "                            return;\n",
              "                        }\n",
              "                        element.appendChild(document.createTextNode(''));\n",
              "                        const url = await google.colab.kernel.proxyPort(port, {cache});\n",
              "\n",
              "                        const external_link = document.createElement('div');\n",
              "                        external_link.innerHTML = `\n",
              "                            <div style=\"font-family: monospace; margin-bottom: 0.5rem\">\n",
              "                                Running on <a href=${new URL(path, url).toString()} target=\"_blank\">\n",
              "                                    https://localhost:${port}${path}\n",
              "                                </a>\n",
              "                            </div>\n",
              "                        `;\n",
              "                        element.appendChild(external_link);\n",
              "\n",
              "                        const iframe = document.createElement('iframe');\n",
              "                        iframe.src = new URL(path, url).toString();\n",
              "                        iframe.height = height;\n",
              "                        iframe.allow = \"autoplay; camera; microphone; clipboard-read; clipboard-write;\"\n",
              "                        iframe.width = width;\n",
              "                        iframe.style.border = 0;\n",
              "                        element.appendChild(iframe);\n",
              "                    })(7862, \"/\", \"100%\", 500, false, window.element)"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Baseline Functionality V2"
      ],
      "metadata": {
        "id": "YAKl-5P4dHEF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# @title\n",
        "with gr.Blocks() as demo:\n",
        "  gr.Markdown(\"# Oral Exam App\")\n",
        "  gr.Markdown(\"## OpenAI API key\")\n",
        "  with gr.Box():\n",
        "    gr.HTML(\"\"\"Embed your OpenAI API key below; if you haven't created one already, visit\n",
        "      platform.openai.com/account/api-keys\n",
        "    to sign up for an account and get your personal API key\"\"\",\n",
        "            elem_classes=\"textbox_label\")\n",
        "    input = gr.Textbox(show_label=False, type=\"password\", container=False,\n",
        "                      placeholder=\"●●●●●●●●●●●●●●●●●\")\n",
        "    input.change(fn=embed_key, inputs=input, outputs=None)\n",
        "\n",
        "  with gr.Blocks():\n",
        "  #########################\n",
        "  #########Context#########\n",
        "  #########################\n",
        "      with gr.Accordion(\"Context section\"):\n",
        "      ### Should also allow vector stores\n",
        "          gr.Markdown(\"## Please upload the context document(s) for Oral exam\")\n",
        "          context_input = gr.File(label=\"Click to upload context file\",\n",
        "                                  file_count=\"multiple\",\n",
        "                                  file_types=[\".txt\", \".docx\", \".pdf\"])\n",
        "          outputs_context=gr.Textbox(label=\"Context\")\n",
        "          context_input.change(fn=process_file, inputs=context_input, outputs=outputs_context)\n",
        "          # upload_button = gr.Button(value=\"Show context\")\n",
        "          # upload_button.click(process_file, context_input, outputs_context)\n",
        "\n",
        "  #########################\n",
        "  ###Question Generation###\n",
        "  #########################\n",
        "      with gr.Accordion(\"Question section\"):\n",
        "          gr.Markdown(\"## Questions\")\n",
        "          with gr.Row():\n",
        "            with gr.Column():\n",
        "              outputs_qa=gr.Textbox(label=\"Generate questions\")\n",
        "              btn1 = gr.Button(value=\"Generate questions\")\n",
        "              btn1.click(generate_questions_v2, inputs=outputs_context, outputs=outputs_qa)\n",
        "\n",
        "  ######################### Need additional work to include these questions when click button   #########################\n",
        "            with gr.Column():\n",
        "              submit_question=gr.Textbox(label=\"Use existing questions\")\n",
        "              btn4 = gr.Button(value=\"Use these questions\")\n",
        "              # btn4.click(use_this_question, inputs=outputs_transcribe, outputs=None)\n",
        "\n",
        "  #########################\n",
        "  #######Main Audio########\n",
        "  #########################\n",
        "      with gr.Accordion(\"Main audio section\"):\n",
        "        gr.Markdown(\"## Upload your audio file or start recording\")\n",
        "        with gr.Column():\n",
        "          ## uploading files seem not working (don't know why)\n",
        "          with gr.Row():\n",
        "              file_input = gr.Audio(label=\"Upload Audio\", source=\"upload\", type=\"filepath\")\n",
        "              record_inputs = gr.Audio(label=\"Record Audio\", source=\"microphone\", type=\"filepath\")\n",
        "\n",
        "          gr.Markdown(\"## Transcribe the audio uploaded or recorded\")\n",
        "          outputs_transcribe=gr.Textbox(label=\"Transcription\")\n",
        "\n",
        "          file_input.change(fn=transcribe, inputs=file_input, outputs=outputs_transcribe)\n",
        "          record_inputs.change(fn=transcribe, inputs=record_inputs, outputs=outputs_transcribe)\n",
        "\n",
        "  #########################\n",
        "  #######Evaluation########\n",
        "  #########################\n",
        "      with gr.Accordion(\"Evaluation section\"):\n",
        "          gr.Markdown(\"## Evaluation\")\n",
        "          with gr.Tab(\"General evalution\"):\n",
        "            evalution=gr.Textbox(label=\"AI Evaluation\")\n",
        "            btn5 = gr.Button(value=\"Evaluate\")\n",
        "            btn5.click(ai_evaluate_v2, inputs=[outputs_context, outputs_transcribe, outputs_qa], outputs=evalution)\n",
        "          with gr.Tab(\"Quantitative evalution\"):\n",
        "            table_output = gr.Dataframe(label = \"Some kind of evaluation metrics?\")\n",
        "            btn6 = gr.Button(value=\"Evaluate\")\n",
        "            btn6.click(ai_evaluate_v2, inputs=[outputs_context, outputs_transcribe, outputs_qa], outputs=table_output)\n",
        "\n",
        "  demo.launch()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 706
        },
        "id": "04KxUQgUcTrm",
        "outputId": "66a9f8c8-36fe-4792-b7d6-3befa6f09269",
        "cellView": "form",
        "collapsed": true
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/gradio/utils.py:833: UserWarning: Expected 2 arguments for function <function generate_questions_v2 at 0x7aa8748f9bd0>, received 1.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/gradio/utils.py:837: UserWarning: Expected at least 2 arguments for function <function generate_questions_v2 at 0x7aa8748f9bd0>, received 1.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Colab notebook detected. To show errors in colab notebook, set debug=True in launch()\n",
            "Note: opening Chrome Inspector may crash demo inside Colab notebooks.\n",
            "\n",
            "To create a public link, set `share=True` in `launch()`.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ],
            "application/javascript": [
              "(async (port, path, width, height, cache, element) => {\n",
              "                        if (!google.colab.kernel.accessAllowed && !cache) {\n",
              "                            return;\n",
              "                        }\n",
              "                        element.appendChild(document.createTextNode(''));\n",
              "                        const url = await google.colab.kernel.proxyPort(port, {cache});\n",
              "\n",
              "                        const external_link = document.createElement('div');\n",
              "                        external_link.innerHTML = `\n",
              "                            <div style=\"font-family: monospace; margin-bottom: 0.5rem\">\n",
              "                                Running on <a href=${new URL(path, url).toString()} target=\"_blank\">\n",
              "                                    https://localhost:${port}${path}\n",
              "                                </a>\n",
              "                            </div>\n",
              "                        `;\n",
              "                        element.appendChild(external_link);\n",
              "\n",
              "                        const iframe = document.createElement('iframe');\n",
              "                        iframe.src = new URL(path, url).toString();\n",
              "                        iframe.height = height;\n",
              "                        iframe.allow = \"autoplay; camera; microphone; clipboard-read; clipboard-write;\"\n",
              "                        iframe.width = width;\n",
              "                        iframe.style.border = 0;\n",
              "                        element.appendChild(iframe);\n",
              "                    })(7863, \"/\", \"100%\", 500, false, window.element)"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def prompt_select(selection, number, length):\n",
        "  if selection == \"Random\":\n",
        "    prompt = f\"Please design a {number} question quiz based on the context provided and the inputted learning objectives (if applicable). The types of questions should be randomized (including multiple choice, short answer, true/false, short answer, etc.).  Provide one question at a time, and wait for my response before providing me with feedback. Again, while the quiz may ask for multiple questions, you should only provide 1 question in you initial response. Do not include the answer in your response. If I get an answer wrong, provide me with an explanation of why it was incorrect, and then give me additional chances to respond until I get the correct choice. Explain why the correct choice is right.\"\n",
        "  elif selection == \"Fill in the Blank\":\n",
        "    prompt = f\"Create a {number} question fill in the blank quiz refrencing the context provided. The quiz should reflect the learning objectives (if inputted). The 'blank' part of the question should appear as '________'. The answers should reflect what word(s) should go in the blank an accurate statement. An example is the follow: 'The author of the article is ______.' The question should be a statement. Provide one question at a time, and wait for my response before providing me with feedback. Again, while the quiz may ask for multiple questions, you should only provide ONE question in you initial response. Do not include the answer in your response. If I get an answer wrong, provide me with an explanation of why it was incorrect,and then give me additional chances to respond until I get the correct choice. Explain why the correct choice is right.\"\n",
        "  elif selection == \"Short Answer\":\n",
        "    prompt = f\"Please design a {number} question quiz about which reflects the learning objectives (if inputted).  The questions should be short answer. Expect the correct answers to be {length} sentences long. Provide one question at a time, and wait for my response before providing me with feedback. Again, while the quiz may ask for multiple questions, you should only provide ONE question in you initial response. Do not include the answer in your response. If I get an answer wrong, provide me with an explanation of why it was incorrect, and then give me additional chances to respond until I get the correct choice. Explain why the correct answer is right.\"\n",
        "  else:\n",
        "    prompt = f\"Please design a {number} question {selection.lower()} quiz based on the context provided and the inputted learning objectives (if applicable). Provide one question at a time, and wait for my response before providing me with feedback. Again, while the quiz may ask for multiple questions, you should only provide 1 question in you initial response. Do not include the answer in your response. If I get an answer wrong, provide me with an explanation of why it was incorrect, and then give me additional chances to respond until I get the correct choice. Explain why the correct choice is right.\"\n",
        "  return prompt\n",
        "\n",
        "\n",
        "# Function to save prompts (premade or custom) and return in the user input box in the chatbot`\n",
        "saved_text = \"\"\n",
        "def save_text(text):\n",
        "    global saved_text\n",
        "    saved_text = text\n",
        "\n",
        "def return_text():\n",
        "    # Return the saved text\n",
        "    return saved_text"
      ],
      "metadata": {
        "id": "wF80F1wU80rU"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Baseline Functionality V3"
      ],
      "metadata": {
        "id": "F5-Ja2evCE4X"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Updated Question Selection and Chatbot Feature"
      ],
      "metadata": {
        "id": "rr8YlzcJCKv4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "with gr.Blocks() as demo:\n",
        "  gr.Markdown(\"# Oral Exam App\")\n",
        "  gr.Markdown(\"## OpenAI API key\")\n",
        "  with gr.Box():\n",
        "    gr.HTML(\"\"\"Embed your OpenAI API key below; if you haven't created one already, visit\n",
        "      platform.openai.com/account/api-keys\n",
        "    to sign up for an account and get your personal API key\"\"\",\n",
        "            elem_classes=\"textbox_label\")\n",
        "    input = gr.Textbox(show_label=False, type=\"password\", container=False,\n",
        "                      placeholder=\"●●●●●●●●●●●●●●●●●\")\n",
        "    input.change(fn=embed_key, inputs=input, outputs=None)\n",
        "\n",
        "  with gr.Blocks():\n",
        "  #########################\n",
        "  #########Context#########\n",
        "  #########################\n",
        "      with gr.Accordion(\"Context section\"):\n",
        "      ### Should also allow vector stores\n",
        "          gr.Markdown(\"## Please upload the context document(s) for Oral exam\")\n",
        "          context_input = gr.File(label=\"Click to upload context file\",\n",
        "                                  file_count=\"multiple\",\n",
        "                                  file_types=[\".txt\", \".docx\", \".pdf\"])\n",
        "          outputs_context=gr.Textbox(label=\"Context\")\n",
        "          context_input.change(fn=process_file, inputs=context_input, outputs=outputs_context)\n",
        "          # upload_button = gr.Button(value=\"Show context\")\n",
        "          # upload_button.click(process_file, context_input, outputs_context)\n",
        "\n",
        "      with gr.Blocks():\n",
        "        gr.Markdown(\"\"\"\n",
        "        ## Generate a Premade Prompt\n",
        "        Select your type and number of desired questions. Click \"Generate Prompt\" to get your premade prompt,\n",
        "        and then \"Insert Prompt into Chat\" to copy the text into the chat interface below. \\\n",
        "        You can also copy the prompt using the icon in the upper right corner and paste directly into the input box when interacting with the model.\n",
        "        \"\"\")\n",
        "        with gr.Row():\n",
        "          with gr.Column():\n",
        "            question_type = gr.Dropdown([\"Multiple Choice\", \"True or False\", \"Short Answer\", \"Fill in the Blank\", \"Random\"], label=\"Question Type\")\n",
        "            number_of_questions = gr.Textbox(label=\"Enter desired number of questions\")\n",
        "            sa_desired_length = gr.Dropdown([\"1-2\", \"3-4\", \"5-6\", \"6 or more\"], label = \"For short answer questions only, choose the desired sentence length for answers. The default value is 1-2 sentences.\")\n",
        "          with gr.Column():\n",
        "            prompt_button = gr.Button(\"Generate Prompt\")\n",
        "            premade_prompt_output = gr.Textbox(label=\"Generated prompt (save or copy)\", show_copy_button=True)\n",
        "            prompt_button.click(prompt_select,\n",
        "                            inputs=[question_type, number_of_questions, sa_desired_length],\n",
        "                            outputs=premade_prompt_output)\n",
        "  ########################\n",
        "  ##Question Generation###\n",
        "  ########################\n",
        "      with gr.Accordion(\"Question section\"):\n",
        "          gr.Markdown(\"## Questions\")\n",
        "          with gr.Row():\n",
        "            with gr.Column():\n",
        "              outputs_qa=gr.Textbox(label=\"Generate questions\")\n",
        "              btn1 = gr.Button(value=\"Generate questions\")\n",
        "              btn1.click(generate_questions_v2, inputs=[outputs_context, premade_prompt_output], outputs=outputs_qa)\n",
        "\n",
        "  ######################### Need additional work to include these questions when click button   #########################\n",
        "            with gr.Column():\n",
        "              submit_question=gr.Textbox(label=\"Use existing questions\")\n",
        "              btn4 = gr.Button(value=\"Use these questions\")\n",
        "              # btn4.click(use_this_question, inputs=outputs_transcribe, outputs=None)\n",
        "\n",
        "  #########################\n",
        "  #######Main Audio########\n",
        "  #########################\n",
        "      with gr.Accordion(\"Main audio section\"):\n",
        "        gr.Markdown(\"## Upload your audio file or start recording\")\n",
        "        with gr.Column():\n",
        "          ## uploading files seem not working (don't know why)\n",
        "          with gr.Row():\n",
        "              file_input = gr.Audio(label=\"Upload Audio\", source=\"upload\", type=\"filepath\")\n",
        "              record_inputs = gr.Audio(label=\"Record Audio\", source=\"microphone\", type=\"filepath\")\n",
        "\n",
        "          gr.Markdown(\"## Transcribe the audio uploaded or recorded\")\n",
        "          outputs_transcribe=gr.Textbox(label=\"Transcription\")\n",
        "\n",
        "          file_input.change(fn=transcribe, inputs=file_input, outputs=outputs_transcribe)\n",
        "          record_inputs.change(fn=transcribe, inputs=record_inputs, outputs=outputs_transcribe)\n",
        "\n",
        "  #########################\n",
        "  #######Evaluation########\n",
        "  #########################\n",
        "      with gr.Accordion(\"Evaluation section\"):\n",
        "          gr.Markdown(\"## Evaluation\")\n",
        "          with gr.Tab(\"General evalution\"):\n",
        "            evalution=gr.Textbox(label=\"AI Evaluation\")\n",
        "            btn5 = gr.Button(value=\"Evaluate\")\n",
        "            btn5.click(ai_evaluate_v2, inputs=[outputs_context, outputs_transcribe, outputs_qa], outputs=evalution)\n",
        "          with gr.Tab(\"Quantitative evalution\"):\n",
        "            table_output = gr.Dataframe(label = \"Some kind of evaluation metrics?\")\n",
        "            btn6 = gr.Button(value=\"Evaluate\")\n",
        "            btn6.click(ai_evaluate_v2, inputs=[outputs_context, outputs_transcribe, outputs_qa], outputs=table_output)\n",
        "\n",
        "\n",
        "      # Chatbot (https://gradio.app/creating-a-chatbot/)\n",
        "      '''\n",
        "      with gr.Blocks():\n",
        "        gr.Markdown(\"\"\"\n",
        "        ## Chat with the Model\n",
        "        Click \"Display Prompt\" to display the premade or custom prompt that you created earlier. Then, continue chatting with the model.\n",
        "        \"\"\")\n",
        "        with gr.Row():\n",
        "          show_prompt_block = gr.Button(\"Display Prompt\")\n",
        "      '''\n",
        "      gr.Markdown(\"## Chat with the Model\")\n",
        "      with gr.Row(equal_height=True):\n",
        "        with gr.Column(scale=2):\n",
        "          chatbot = gr.Chatbot()\n",
        "          with gr.Row():\n",
        "            user_chat_input = gr.Textbox(label=\"User input\", scale=9)\n",
        "            user_chat_input.submit(return_text, inputs=None, outputs=user_chat_input)\n",
        "            user_chat_submit = gr.Button(\"Ask/answer model\", scale=1)\n",
        "            #show_prompt_block.click(return_text, inputs=None, outputs=user_chat_input)\n",
        "\n",
        "        # TODO Move the sources so it's displayed to the right of the chat bot,\n",
        "        # with the sources taking up about 1/3rd of the horizontal space\n",
        "        # with gr.Box(elem_id=\"sources-container\", scale=1):\n",
        "        #   # TODO: Display document sources in a nicer format?\n",
        "        #   gr.HTML(value=\"<h3 id='sources'>Sources</h3>\")\n",
        "        #   sources_output = []\n",
        "        #   for i in range(num_sources):\n",
        "        #     source_elem = gr.HTML(visible=False)\n",
        "        #     sources_output.append(source_elem)\n",
        "\n",
        "demo.launch()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 616
        },
        "id": "Y7-3JFuZ8H5k",
        "outputId": "ea99ce65-7b79-4d39-dd88-44785b0d6615"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Colab notebook detected. To show errors in colab notebook, set debug=True in launch()\n",
            "Note: opening Chrome Inspector may crash demo inside Colab notebooks.\n",
            "\n",
            "To create a public link, set `share=True` in `launch()`.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ],
            "application/javascript": [
              "(async (port, path, width, height, cache, element) => {\n",
              "                        if (!google.colab.kernel.accessAllowed && !cache) {\n",
              "                            return;\n",
              "                        }\n",
              "                        element.appendChild(document.createTextNode(''));\n",
              "                        const url = await google.colab.kernel.proxyPort(port, {cache});\n",
              "\n",
              "                        const external_link = document.createElement('div');\n",
              "                        external_link.innerHTML = `\n",
              "                            <div style=\"font-family: monospace; margin-bottom: 0.5rem\">\n",
              "                                Running on <a href=${new URL(path, url).toString()} target=\"_blank\">\n",
              "                                    https://localhost:${port}${path}\n",
              "                                </a>\n",
              "                            </div>\n",
              "                        `;\n",
              "                        element.appendChild(external_link);\n",
              "\n",
              "                        const iframe = document.createElement('iframe');\n",
              "                        iframe.src = new URL(path, url).toString();\n",
              "                        iframe.height = height;\n",
              "                        iframe.allow = \"autoplay; camera; microphone; clipboard-read; clipboard-write;\"\n",
              "                        iframe.width = width;\n",
              "                        iframe.style.border = 0;\n",
              "                        element.appendChild(iframe);\n",
              "                    })(7864, \"/\", \"100%\", 500, false, window.element)"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### What's left\n",
        "- vector store (link) upload\n",
        "- submit question section need to be linked with ai_evaluate function"
      ],
      "metadata": {
        "id": "g2EVIogW69Fd"
      }
    }
  ]
}