{ "cells": [ { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
doc_idcontentsmetadata
0f0567fea-1193-4223-a826-c0016297c3a7240731,집행에관한이의,2023그887,2024.04.30,대법원,민사,4001...{'last_modified_datetime': 2024-07-11 23:32:01...
16ac682e2-c8c1-4f61-a0e3-1c5f4f42e8e2240739,손해배상(기)[영국법상 채무불이행에 대한 위약금 약정이 손해배상액의 예...{'last_modified_datetime': 2024-07-11 23:32:01...
2fd68837f-c29a-4ed9-ac6a-30b00f06333b240743,구상금[도시 일용근로자의 월 가동일수가 문제된 사건],2020다2716...{'last_modified_datetime': 2024-07-11 23:32:01...
35c307884-bcf3-479d-9693-70c967a8878a240735,벌점부과처분취소[건설기술 진흥법 제53조 제1항에서 규정한 벌점부과처분...{'last_modified_datetime': 2024-07-11 23:32:01...
430cc8063-071d-4c41-918f-ded6ea67b622240747,손해배상(국)[문서제출명령에대한재항고][금융감독원 직원이 직무상 작성하...{'last_modified_datetime': 2024-07-11 23:32:01...
\n", "
" ], "text/plain": [ " doc_id \\\n", "0 f0567fea-1193-4223-a826-c0016297c3a7 \n", "1 6ac682e2-c8c1-4f61-a0e3-1c5f4f42e8e2 \n", "2 fd68837f-c29a-4ed9-ac6a-30b00f06333b \n", "3 5c307884-bcf3-479d-9693-70c967a8878a \n", "4 30cc8063-071d-4c41-918f-ded6ea67b622 \n", "\n", " contents \\\n", "0 240731,집행에관한이의,2023그887,2024.04.30,대법원,민사,4001... \n", "1 240739,손해배상(기)[영국법상 채무불이행에 대한 위약금 약정이 손해배상액의 예... \n", "2 240743,구상금[도시 일용근로자의 월 가동일수가 문제된 사건],2020다2716... \n", "3 240735,벌점부과처분취소[건설기술 진흥법 제53조 제1항에서 규정한 벌점부과처분... \n", "4 240747,손해배상(국)[문서제출명령에대한재항고][금융감독원 직원이 직무상 작성하... \n", "\n", " metadata \n", "0 {'last_modified_datetime': 2024-07-11 23:32:01... \n", "1 {'last_modified_datetime': 2024-07-11 23:32:01... \n", "2 {'last_modified_datetime': 2024-07-11 23:32:01... \n", "3 {'last_modified_datetime': 2024-07-11 23:32:01... \n", "4 {'last_modified_datetime': 2024-07-11 23:32:01... " ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "corpus_df = pd.read_parquet(\"evaluation_data/corpus.parquet\", engine=\"pyarrow\")\n", "corpus_df.head()" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "prompt = \"\"\"다음은 법률 판례에 관한 내용입니다.\n", "내용을 보고 법률 관련 질문을 만드세요.\n", "반드시 주어진 판례와 관련한 질문이어야 합니다.\n", "질문 형식는 질문자가 실제 사례를 설명하고 비슷한 판례를 찾아달라는 질문입니다.\n", "만약 주어진 내용이 법률 판례와 관련되지 않았다면,\n", "'법률 판례와 관련 없습니다.'라고 질문을 만드세요.\n", "\n", "판례 내용:\n", "{{text}}\n", "\n", "생성할 질문 개수: {{num_questions}}\n", "\n", "예시:\n", "[Q]: 중고차 거래를 했는데 불량으로 차 수리에 500만원이 들었습니다. 판매자에게 법적 책임을 물을 수 있나요? 비슷한 판례를 소개해주세요.\n", "[A]: 자동차 사기와 관련된 최근 판례를 세 가지 소개해드리겠습니다. \n", "\n", "법률 판례와 관련이 없는 내용일 경우 예시:\n", "[Q]: 법률 판례와 관련 없습니다.\n", "[A]: 법률 판례와 관련 없습니다.\n", "\n", "결과:\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "from dotenv import load_dotenv\n", "from llama_index.llms.upstage import Upstage\n", "\n", "load_dotenv()\n", "\n", "llm = Upstage(temperature=1.0, max_retries=100)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "0it [00:00, ?it/s]" ] }, { "data": { "text/html": [ "
[07/11/24 23:52:21] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:21]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=779274;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=753072;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:22] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:22]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=620763;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=792294;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
                    INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=72700;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=707471;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:24] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:24]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=830154;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=377811;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:25] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:25]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=853232;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=721124;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:26] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:26]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=938920;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=524042;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:28] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:28]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=17780;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=88802;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
                    INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=893838;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=887881;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
                    INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=287839;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=532736;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:30] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:30]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=469851;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=965758;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:32] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:32]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=920847;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=424018;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:33] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:33]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=768867;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=978022;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:34] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:34]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=903128;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=27618;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:35] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:35]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=50897;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=92282;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:42] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:42]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=339037;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=740758;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:45] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:45]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=778828;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=744469;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:47] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:47]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=465891;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=475010;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:48] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:48]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=300901;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=730434;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:51] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:51]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=798318;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=280791;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:52] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:52]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=484999;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=163910;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:53] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:53]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=964588;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=397415;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
                    INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=552211;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=883806;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
                    INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=92138;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=708308;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
                    INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=926481;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=783387;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
[07/11/24 23:52:54] INFO     [_client.py:1773] >> HTTP Request: POST                                _client.py:1773\n",
       "                             https://api.upstage.ai/v1/solar/chat/completions \"HTTP/1.1 200 OK\"                    \n",
       "
\n" ], "text/plain": [ "\u001b[2;36m[07/11/24 23:52:54]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO \u001b[0m \u001b[1m[\u001b[0m_client.py:\u001b[1;36m1773\u001b[0m\u001b[1m]\u001b[0m >> HTTP Request: \u001b[1;33mPOST\u001b[0m \u001b]8;id=116090;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py\u001b\\\u001b[2m_client.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=752469;file:///Users/anpigon/Library/Caches/pypoetry/virtualenvs/law-bot-C3zMZhS7-py3.11/lib/python3.11/site-packages/httpx/_client.py#1773\u001b\\\u001b[2m1773\u001b[0m\u001b]8;;\u001b\\\n", "\u001b[2;36m \u001b[0m \u001b[4;94mhttps://api.upstage.ai/v1/solar/chat/completions\u001b[0m \u001b[32m\"HTTP/1.1 200 OK\"\u001b[0m \u001b[2m \u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "1it [00:35, 35.05s/it]\n" ] } ], "source": [ "import nest_asyncio\n", "from autorag.data.qacreation import generate_qa_llama_index, make_single_content_qa\n", "\n", "nest_asyncio.apply()\n", "\n", "qa_df = make_single_content_qa(\n", " corpus_df,\n", " content_size=20,\n", " qa_creation_func=generate_qa_llama_index,\n", " llm=llm,\n", " prompt=prompt,\n", " question_num_per_content=1,\n", ")" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "qa_df = qa_df.loc[~qa_df[\"query\"].str.contains(\"법률 판례와 관련 없습니다.\")]\n", "qa_df.reset_index(drop=True, inplace=True)\n", "qa_df.to_parquet(\"evaluation_data/qa.parquet\")" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "qa_df.to_excel(\"evaluation_data/qa.xlsx\", index=False)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "RangeIndex: 20 entries, 0 to 19\n", "Data columns (total 4 columns):\n", " # Column Non-Null Count Dtype \n", "--- ------ -------------- ----- \n", " 0 retrieval_gt 20 non-null object\n", " 1 qid 20 non-null object\n", " 2 query 20 non-null object\n", " 3 generation_gt 20 non-null object\n", "dtypes: object(4)\n", "memory usage: 772.0+ bytes\n" ] } ], "source": [ "qa_df.info()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "law-bot-C3zMZhS7-py3.11", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.9" } }, "nbformat": 4, "nbformat_minor": 2 }