Spaces:
Sleeping
Sleeping
kz209
commited on
Commit
•
29fb045
1
Parent(s):
203771e
add test_data.json and update batch evaluation
Browse files- build_json.ipynb +251 -0
- pages/batch_evaluation.py +12 -14
- test_samples/test_data.json +1 -0
build_json.ipynb
ADDED
@@ -0,0 +1,251 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"cells": [
|
3 |
+
{
|
4 |
+
"cell_type": "code",
|
5 |
+
"execution_count": 1,
|
6 |
+
"metadata": {},
|
7 |
+
"outputs": [],
|
8 |
+
"source": [
|
9 |
+
"import json\n",
|
10 |
+
"\n",
|
11 |
+
"a = [\n",
|
12 |
+
" {\n",
|
13 |
+
" \"dialogue\": \"\"\"[doctor] hi michelle what's been going on the medical assistant told me that you injured your knee\n",
|
14 |
+
"[patient] yeah i'm gon na have to keep this my favorite story is that i had a sandra who injured her knee cold dancing so that would immediately came up to mine but let's try skiing so i was skiing in vermont last weekend and i caught my ski when i was coming off the lift and i fell and my knee popped and it's hurt ever since\n",
|
15 |
+
"[doctor] okay and were you able to ski down the hill or did you have to be taken down by the ski patrol\n",
|
16 |
+
"[patient] i had to be taken down by the ski patrol i was unable to bear weight\n",
|
17 |
+
"[doctor] okay is n't that slightly terrifying they had to do the same thing for me and you're just kind of sitting there watching all of the trees go by so anyway so what part of your knee is bothering you the most is it the inside the outside\n",
|
18 |
+
"[patient] it hurts on the outside but it also feels like it wo n't hold my weight\n",
|
19 |
+
"[doctor] okay so it feels like it's gon na buckle\n",
|
20 |
+
"[patient] hmmm yes\n",
|
21 |
+
"[doctor] okay alright and are you able to bend it\n",
|
22 |
+
"[patient] i was immediately after the injury but not really now\n",
|
23 |
+
"[doctor] okay and how about straightening it are you able to straighten it at all\n",
|
24 |
+
"[patient] somewhat but it really hurts when i try to straighten all the way\n",
|
25 |
+
"[doctor] okay alright now do you have any numbing or tingling in your toes\n",
|
26 |
+
"[patient] no\n",
|
27 |
+
"[doctor] okay and can you feel your toes okay\n",
|
28 |
+
"[patient] i think so\n",
|
29 |
+
"[doctor] okay alright now are you a pretty active person you said you were skiing do you do what kind of other activities do you do\n",
|
30 |
+
"[patient] i chase my two four -year-old\n",
|
31 |
+
"[doctor] okay do you have twins\n",
|
32 |
+
"[patient] i do\n",
|
33 |
+
"[doctor] nice that's great\n",
|
34 |
+
"[patient] yeah\n",
|
35 |
+
"[doctor] so okay and what's your support like at home who is who is able to help out are you still doing that kind of hobbling around on crutches or\n",
|
36 |
+
"[patient] yeah now my husband is home with me but he also works a job where he is gone quite a bit so it's just me and the kids\n",
|
37 |
+
"[doctor] okay alright and did the ski patrol give you crutches i i see you have them here with you okay alright great and are you otherwise generally healthy\n",
|
38 |
+
"[patient] yes\n",
|
39 |
+
"[doctor] okay and no high blood pressure diabetes anything like that\n",
|
40 |
+
"[patient] mm-hmm\n",
|
41 |
+
"[doctor] no okay alright and well i i wan na go ahead let me just do a quick physical exam i'm gon na be calling out some of my exam findings and i'll let you know what that means when i'm done okay alright so on your heart exam your heart sounds in a nice regular rate and rhythm i do n't appreciate any murmur on your lung exam your lungs are nice and clear to auscultation and remind me what knee did you injure again\n",
|
42 |
+
"[patient] my right knee\n",
|
43 |
+
"[doctor] your right knee okay so on your right knee examination i do appreciate some edema and an effusion over the right knee does it hurt when i press\n",
|
44 |
+
"[patient] yes\n",
|
45 |
+
"[doctor] okay there is pain to palpation of the right lateral knee there is decreased flexion and extension there is a positive lachman sign there is a palpable palpable dorsalis pedis and posterior tibial pulse there is no leg edema in the ankle okay well let's just talk a little bit about you know my assessment and you know my plan for you so you know i know that you had the x-ray done of your of your right knee that did n't show any bony abnormality but i i'm concerned that you have ruptured your your acl or your anterior cruciate ligament that's like a major ligament that helps connect and helps your knee move back and forth so i wan na go ahead and order a knee mri just so that we can get a a a good look and just you know confirm that physical exam okay now some people can have a normal physical exam and their acl can still be torn but you do have a lot of pain on the lateral aspect of your knee so i wan na make sure if there make sure that there is not any other structures that have been damaged by this accident okay i wan na go ahead and you know are you what are you taking for the pain\n",
|
46 |
+
"[patient] ibuprofen\n",
|
47 |
+
"[doctor] is that helping\n",
|
48 |
+
"[patient] somewhat yes\n",
|
49 |
+
"[doctor] okay alright do you want something stronger\n",
|
50 |
+
"[patient] no i'm okay\n",
|
51 |
+
"[doctor] alright so let's just continue with ibuprofen you can take you know six hundred to eight hundred milligrams every eight hours as needed i wan na go ahead and put you in a brace that's gon na help your knee feel a bit more supported okay and let's go ahead and i'm hoping that we can get this mri done in you know the next couple days and then we can have a conversation about what needs to be done now you said that you are are are you know obviously a very active active mom any other exercise or anything else that that i should be aware of that you do just wondering in terms of which kind of graft we would use to fix this with your with your acl being injured\n",
|
52 |
+
"[patient] hmmm no i not i i would like to get back to the running but currently not that active\n",
|
53 |
+
"[doctor] okay alright alright do you have any questions about anything\n",
|
54 |
+
"[patient] i think you've explained it well\n",
|
55 |
+
"[doctor] okay alright so i'll see you again soon okay\n",
|
56 |
+
"[patient] okay thank you\n",
|
57 |
+
"[doctor] alright take care bye\"\"\",\n",
|
58 |
+
" \"summary\": \"\"\"CHIEF COMPLAINT\n",
|
59 |
+
"\n",
|
60 |
+
"Right knee injury\n",
|
61 |
+
"\n",
|
62 |
+
"HISTORY OF PRESENT ILLNESS\n",
|
63 |
+
"\n",
|
64 |
+
"Michelle King is a pleasant 44-year-old male who presents to the clinic today for the evaluation of a right knee injury. Her injury was sustained while skiing in Vermont last weekend after she caught her ski when coming off of the lift and fell. She reports a pop in her knee at the time of injury and has been experiencing pain since that time. Due to her fall she was unable to weight bear and had to be taken down by the ski patrol. She locates her pain to the lateral aspect of her knee and also describes the feeling of instability as well as buckling. At this time she is experiencing difficulty with knee flexion; however, she states this was not a problem immediately after the injury. Full knee extension is also quite painful for her. The patient denies any numbness or tingling in her toes. She is currently utilizing crutches for ambulation which were provided to her by the ski patrol. In terms of pain control, she is taking ibuprofen which is providing some relief.\n",
|
65 |
+
"\n",
|
66 |
+
"MEDICAL HISTORY\n",
|
67 |
+
"\n",
|
68 |
+
"Patient reports she is otherwise healthy and denies a history of high blood pressure or diabetes.\n",
|
69 |
+
"\n",
|
70 |
+
"SOCIAL HISTORY\n",
|
71 |
+
"\n",
|
72 |
+
"Patient reports she is very active with skiing and taking care of her 4-year-old twins. She mentions that she would like to get back into running at some point. She states that her husband is home with her, but he also works a job where he has gone quite a bit.\n",
|
73 |
+
"\n",
|
74 |
+
"MEDICATIONS\n",
|
75 |
+
"\n",
|
76 |
+
"Patient reports she is taking ibuprofen.\n",
|
77 |
+
"\n",
|
78 |
+
"REVIEW OF SYSTEMS\n",
|
79 |
+
"\n",
|
80 |
+
"Musculoskeletal: Reports right knee pain and limited range of motion.\n",
|
81 |
+
"Neurological: Denies numbness or tingling in toes.\n",
|
82 |
+
"\n",
|
83 |
+
"PHYSICAL EXAM\n",
|
84 |
+
"\n",
|
85 |
+
"GAIT: The patient is ambulating with crutches.\n",
|
86 |
+
"CV: Regular rate and rhythm. No murmur.\n",
|
87 |
+
"RESPIRATORY: Lungs are clear to auscultation.\n",
|
88 |
+
"MSK: Examination of the right knee: Edema and effusion noted. Pain with palpation of the lateral knee. Decreased flexion and extension. Positive Lachman's. Palpable dorsalis pedis and posterior tibial pulse. No ankle edema.\n",
|
89 |
+
"\n",
|
90 |
+
"RESULTS\n",
|
91 |
+
"\n",
|
92 |
+
"4 views of the right knee were taken today. These reveal no bony abnormalities.\n",
|
93 |
+
"\n",
|
94 |
+
"ASSESSMENT\n",
|
95 |
+
"\n",
|
96 |
+
"Right knee pain, possible ACL tear.\n",
|
97 |
+
"\n",
|
98 |
+
"PLAN\n",
|
99 |
+
"\n",
|
100 |
+
"After reviewing the patient's examination and radiographic findings today, I have had a lengthy discussion with the patient in regards to her current symptoms. I have recommended that we obtain an MRI of the right knee to evaluate for a possible ACL tear. In the meantime, I have advised her to continue taking ibuprofen 600 to 800 mg every 8 hours as needed for pain. She will also be placed into a brace for increased knee support.\n",
|
101 |
+
"\n",
|
102 |
+
"INSTRUCTIONS\n",
|
103 |
+
"\n",
|
104 |
+
"The patient will follow up with me once the MRI results are available for review and further discussion.\"\"\",\n",
|
105 |
+
"\"format\": \"section\"\n",
|
106 |
+
" },{\n",
|
107 |
+
" \"dialogue\": \"\"\"[doctor] okay hi wayne well i understand you're here for you've got a sore on your foot that's not healing is that right\n",
|
108 |
+
"[patient] yes\n",
|
109 |
+
"[doctor] so can you tell me about that how are you doing\n",
|
110 |
+
"[patient] well i've been doing okay but i've had this wound on my right foot for a couple of weeks and it's not getting better i saw my pcp and they referred me to you i i used to see a podiatrist and a couple of years ago but they moved and i was n't able to get another one\n",
|
111 |
+
"[doctor] okay and how long have you had the wound\n",
|
112 |
+
"[patient] about two to three weeks\n",
|
113 |
+
"[doctor] okay have you had any kind of trauma to that foot\n",
|
114 |
+
"[patient] no i bought a new pair of shoes to travel with and the neuropathy i guess i did n't feel that they were too tight at first it was just a blister but it looked a lot worse now i've been putting a band-aid on it and\n",
|
115 |
+
"[doctor] okay and so did your doctor put you on any antibiotics when they they saw your foot\n",
|
116 |
+
"[patient] yes i finished the course yesterday the wound looked about the same though\n",
|
117 |
+
"[doctor] okay now does it hurt\n",
|
118 |
+
"[patient] no i ca n't feel it\n",
|
119 |
+
"[doctor] okay now have you had any other symptoms like fever chills drainage from the wound or anything along those lines\n",
|
120 |
+
"[patient] no but when i take my sock off sometimes it sticks to the wound even when i put a band-aid on\n",
|
121 |
+
"[doctor] okay well are you still wearing those same shoes that cause the problem\n",
|
122 |
+
"[patient] no i'm upset\n",
|
123 |
+
"[doctor] i'm so upset too i love those shoes\n",
|
124 |
+
"[patient] no i'm so upset i i love those shoes\n",
|
125 |
+
"[doctor] okay so what kind of shoes are they\n",
|
126 |
+
"[patient] they're hoka's\n",
|
127 |
+
"[doctor] ah so where did you get them\n",
|
128 |
+
"[patient] i got them at rei couple of years ago\n",
|
129 |
+
"[doctor] they're pretty good about helping you to fit your fit you in a shoe are n't they\n",
|
130 |
+
"[patient] yeah\n",
|
131 |
+
"[doctor] so what do you like about them\n",
|
132 |
+
"[patient] they are comfortable they are easy to take on and off and they provide good stability\n",
|
133 |
+
"[doctor] that's good so you've had some issues with stability sometimes or\n",
|
134 |
+
"[patient] yes\n",
|
135 |
+
"[doctor] okay okay well another question i want to ask you is how is your diabetes doing\n",
|
136 |
+
"[patient] i do n't think it's too bad my last hgb a1c was a little over eight\n",
|
137 |
+
"[doctor] okay alright well let me just do a quick physical exam okay for vital signs your temperature is ninety eight . one your vital signs look good your heart rate is seventy two respirations sixteen blood pressure is one ten over sixty five okay so on your foot exam let's see there is a one by two inch circular wound on the dorsal aspect of the lateral right foot it is just proximal to the right fifth to the fifth mtp joint and there is some yellow slough present with minimal granulation tissue there's no surrounding erythema or cellulitis and there's no evidence of fluid collection there's no necrosis there is no odor i do not appreciate any bony exposure on on vascular exam there are palpable bilateral femoral and popliteal pulses there are no palpable dp or pt pulses but doppler signs are present okay so does this hurt when i touch it here\n",
|
138 |
+
"[patient] no it's okay\n",
|
139 |
+
"[doctor] okay alright so i've reviewed the results your right foot x-ray that we did before you i came in the room and this shows no evidence of osteomyelitis which means there is no evidence of bone infection so that's really good so let me tell you a little about my assessment and plan for you so for your first problem your diabetic foot ulcer so i want to order an ankle brachial index abi to determine the blood supply to your foot to see if you can heal this wound i'm also gon na perform a debridement here in the office to take off some of the dead tissue and then next i'm going to prescribe a collagenase ointment to be applied to the wound once daily and then cover with a dry sterile dressing now we will continue this until we see the wound shrinking and a nice pink tissue is present and i want you to wear a surgical shoe to take pressure off of the area okay so i do n't think any more antibiotics are needed at this time and i want to see you again in two weeks so we may need to refer you to a vascular specialist if the abi indicates your blood supply is not optimal for wound healing okay\n",
|
140 |
+
"[patient] okay\n",
|
141 |
+
"[doctor] okay so for your diabetes it's gon na be very important to get your diabetes under control in order to get the wound to heal so we might even go ahead and get a diabetic consult so that we can go over some some maybe some ways you can modify your diet without a being too much but maybe that can bring down your hemoglobin a1c into a little bit better level okay\n",
|
142 |
+
"[patient] okay\n",
|
143 |
+
"[doctor] alright do you have any other questions\n",
|
144 |
+
"[patient] no\n",
|
145 |
+
"[doctor] okay\"\"\",\n",
|
146 |
+
"\"summary\": \"\"\"HISTORY OF PRESENT ILLNESS\n",
|
147 |
+
"\n",
|
148 |
+
"Wayne Jenkins is a pleasant 43-year-old male who presents to the clinic today for the evaluation of a right foot wound, onset 2 to 3 weeks ago. He was seen by his primary care physician who referred him to our office. His last visit with a podiatrist was 2 years ago, however they moved, and he has been unable to get another podiatrist. He denies any specific injury.\n",
|
149 |
+
"\n",
|
150 |
+
"The patient purchased a new pair of shoes and was breaking them in prior to traveling. Initially, he did not feel that the shoes were too tight, however a blister formed on his right foot. The patient does have neuropathy and notes this might be why he did not notice the tightness. His blister soon became a wound and he was seen by his PCP. He was placed on a round of antibiotics, which he completed yesterday. He denies any changes to the appearance of the wound. The patient denies pain or drainage. His sock will occasionally stick to his wound, regardless of if he has a Band-Aid covering it. The patient has discontinued the use of these shoes. Occasionally, he has issues with stability, which was a primary purpose of purchasing these shoes. He denies any fever or chills.\n",
|
151 |
+
"\n",
|
152 |
+
"Regarding his diabetes, his last hemoglobin A1C was 8.\n",
|
153 |
+
"\n",
|
154 |
+
"REVIEW OF SYSTEMS\n",
|
155 |
+
"\n",
|
156 |
+
"Constitutional: Denies fever and chills.\n",
|
157 |
+
"Skin: Reports right foot wound.\n",
|
158 |
+
"Neurological: Reports neuropathy.\n",
|
159 |
+
"\n",
|
160 |
+
"VITALS\n",
|
161 |
+
"\n",
|
162 |
+
"Temperature: 98.1.\n",
|
163 |
+
"Heart rate: 72.\n",
|
164 |
+
"Respiration: 16.\n",
|
165 |
+
"Blood pressure: 110/65 mm Hg.\n",
|
166 |
+
"\n",
|
167 |
+
"PHYSICAL EXAM\n",
|
168 |
+
"\n",
|
169 |
+
"CV: Palpable bilateral femoral and popliteal pulses. No palpable DP or PT pulses but Doppler signs are present. No evidence of fluid collection.\n",
|
170 |
+
"SKIN: On the right foot, there is a 1 x 2 inch circular wound on the dorsal aspect of the lateral right foot that is just proximal to the 5th MTP joint. There is some yellow slough present with minimal granulation tissue. No surrounding erythema or cellulitis. No necrosis. No odor.\n",
|
171 |
+
"\n",
|
172 |
+
"RESULTS\n",
|
173 |
+
"\n",
|
174 |
+
"3 views of the right foot were taken today. These reveal no evidence of osteomyelitis.\n",
|
175 |
+
"\n",
|
176 |
+
"ASSESSMENT\n",
|
177 |
+
"\n",
|
178 |
+
"1. Right foot diabetic ulcer.\n",
|
179 |
+
"2. Diabetes.\n",
|
180 |
+
"\n",
|
181 |
+
"PLAN\n",
|
182 |
+
"\n",
|
183 |
+
"After reviewing the patient's examination and radiographic findings today, I have had a lengthy discussion with the patient regarding his current symptoms. I have explained to him that his x-rays did not reveal any evidence of osteomyelitis. At this time, I have recommended that we obtain an ankle brachial index ABI to determine the blood supply to his foot to see if he can heal this wound. I have also performed a debridement in the office to remove some of the dead tissue. I have prescribed a collagenase ointment to be applied to the wound 1 time daily and instructed him to cover with a dry sterile dressing. He will continue this until we see the wound shrinking and the nice pink tissue is present. I have also recommended that the patient be placed in a surgical shoe to take pressure off the area. I do not think that antibiotics are needed at this time. He may need to see a vascular specialist if the ABI indicates that his blood supply is not optimal for wound healing.\n",
|
184 |
+
"\n",
|
185 |
+
"Regarding his diabetes, I explained the need for controlling his diabetes and the effects this will have on wound healing. I have recommended a diabetic consultation to discuss diet modifications to bring his hemoglobin A1c to a better level.\n",
|
186 |
+
"\n",
|
187 |
+
"INSTRUCTIONS\n",
|
188 |
+
"\n",
|
189 |
+
"The patient will follow up with me in 2 weeks to check on his progress.\"\"\",\n",
|
190 |
+
"\"format\": \"section\"\n",
|
191 |
+
" }, {\n",
|
192 |
+
" \"dialogue\": \"\"\"Doctor: How old are you young man?\n",
|
193 |
+
"Patient: I am nineteen. \n",
|
194 |
+
"Doctor: What happened? How did you get hurt?\n",
|
195 |
+
"Patient: I was in a fight. Some people are just rude. \n",
|
196 |
+
"Doctor: Oh, that's bad! How did you hurt your right hand?\n",
|
197 |
+
"Patient: Ah, I punched that other guy in face or maybe his mouth, I don't remember exactly but he got a good one. Later, in anger I punched in a glass. It broke the glass and spilled all the drink. I think I got a cut as it was bleeding badly. \n",
|
198 |
+
"Doctor: I see three cuts in your right hand. \n",
|
199 |
+
"Doctor: When you came here in Emergency your wound was clean and looked like somebody tried to fix your cut. Do you remember any details about it?\n",
|
200 |
+
"Patient: Yeah, I remember there was a navy corpsman. He was very helpful; he cleaned my cut with the tap water. He also tried to close it up with gauze and superglue. He said it will stop bleeding, but nothing helped. They called for help and here I am in emergency.\"\"\",\n",
|
201 |
+
"\"summary\": \"\"\"The patient is a 19-year-old male who was involved in a fight approximately an hour prior to his ED presentation. He punched a guy few times on the face, might be the mouth and then punched a drinking glass, breaking it and lacerating his right hand. He has three lacerations on his right hand. His wound was cleaned out thoroughly with tap water, and one of the navy corpsman tried to use Superglue and gauze to repair it. However, it continues to bleed and he is here for evaluation\"\"\",\n",
|
202 |
+
"\"format\": \"free-form\"\n",
|
203 |
+
" }, {\n",
|
204 |
+
" \"dialogue\": \"\"\"Doctor: What medications are you taking currently? \n",
|
205 |
+
"Patient: Well, I'm taking Remeron for depression and Ziac for high blood pressure.\"\"\",\n",
|
206 |
+
"\"summary\": \"\"\"His only medications are Ziac and Remeron.\"\"\",\n",
|
207 |
+
"\"format\": \"\"\"free-form\"\"\"\n",
|
208 |
+
" }\n",
|
209 |
+
"]"
|
210 |
+
]
|
211 |
+
},
|
212 |
+
{
|
213 |
+
"cell_type": "code",
|
214 |
+
"execution_count": 2,
|
215 |
+
"metadata": {},
|
216 |
+
"outputs": [],
|
217 |
+
"source": [
|
218 |
+
"with open('/Users/yukino/Downloads/HF_space/Chris-lab/test_samples/test_data.json', 'w') as f:\n",
|
219 |
+
" json.dump(a, f)"
|
220 |
+
]
|
221 |
+
},
|
222 |
+
{
|
223 |
+
"cell_type": "code",
|
224 |
+
"execution_count": null,
|
225 |
+
"metadata": {},
|
226 |
+
"outputs": [],
|
227 |
+
"source": []
|
228 |
+
}
|
229 |
+
],
|
230 |
+
"metadata": {
|
231 |
+
"kernelspec": {
|
232 |
+
"display_name": "Python 3",
|
233 |
+
"language": "python",
|
234 |
+
"name": "python3"
|
235 |
+
},
|
236 |
+
"language_info": {
|
237 |
+
"codemirror_mode": {
|
238 |
+
"name": "ipython",
|
239 |
+
"version": 3
|
240 |
+
},
|
241 |
+
"file_extension": ".py",
|
242 |
+
"mimetype": "text/x-python",
|
243 |
+
"name": "python",
|
244 |
+
"nbconvert_exporter": "python",
|
245 |
+
"pygments_lexer": "ipython3",
|
246 |
+
"version": "3.11.9"
|
247 |
+
}
|
248 |
+
},
|
249 |
+
"nbformat": 4,
|
250 |
+
"nbformat_minor": 2
|
251 |
+
}
|
pages/batch_evaluation.py
CHANGED
@@ -1,14 +1,14 @@
|
|
1 |
from dotenv import load_dotenv
|
2 |
import gradio as gr
|
3 |
-
|
|
|
4 |
|
5 |
import numpy as np
|
6 |
|
7 |
from utils.model import Model
|
8 |
-
from utils.data import dataset
|
9 |
from utils.metric import metric_rouge_score
|
10 |
|
11 |
-
from pages.summarization_playground import
|
12 |
|
13 |
load_dotenv()
|
14 |
|
@@ -45,14 +45,14 @@ def display_results(response_list):
|
|
45 |
|
46 |
return html_output
|
47 |
|
48 |
-
def process(
|
49 |
-
random.seed(seed)
|
50 |
response_list = []
|
51 |
-
|
52 |
-
for data in
|
53 |
dialogue = data['dialogue']
|
|
|
54 |
summary = data['summary']
|
55 |
-
response = generate_answer(dialogue, model_selection, prompt)
|
56 |
|
57 |
rouge_score = metric_rouge_score(response, summary)
|
58 |
|
@@ -72,18 +72,16 @@ def process(seed, model_selection, prompt, num=10):
|
|
72 |
|
73 |
def create_batch_evaluation_interface():
|
74 |
with gr.Blocks() as demo:
|
75 |
-
gr.Markdown("## Here are evaluation setups. It will
|
76 |
-
|
77 |
-
|
78 |
-
seed = gr.Number(value=8, info="pick your favoriate random seed", precision=0)
|
79 |
-
model_dropdown = gr.Dropdown(choices=Model.__model_list__, label="Choose a model", value=Model.__model_list__[0])
|
80 |
Template_text = gr.Textbox(value="""Summarize the following dialogue""", label='Input Prompting Template', lines=8, placeholder='Input your prompts')
|
81 |
submit_button = gr.Button("✨ Submit ✨")
|
82 |
output = gr.HTML(label="Results")
|
83 |
|
84 |
submit_button.click(
|
85 |
process,
|
86 |
-
inputs=[
|
87 |
outputs=output
|
88 |
)
|
89 |
|
|
|
1 |
from dotenv import load_dotenv
|
2 |
import gradio as gr
|
3 |
+
|
4 |
+
import json
|
5 |
|
6 |
import numpy as np
|
7 |
|
8 |
from utils.model import Model
|
|
|
9 |
from utils.metric import metric_rouge_score
|
10 |
|
11 |
+
from pages.summarization_playground import generate_answer
|
12 |
|
13 |
load_dotenv()
|
14 |
|
|
|
45 |
|
46 |
return html_output
|
47 |
|
48 |
+
def process(model_selection, prompt, num=10):
|
|
|
49 |
response_list = []
|
50 |
+
dataset = json.load("test_samples/test_data.json")
|
51 |
+
for data in dataset:
|
52 |
dialogue = data['dialogue']
|
53 |
+
format = data['format']
|
54 |
summary = data['summary']
|
55 |
+
response = generate_answer(dialogue, model_selection, prompt + f' Output following {format} format.')
|
56 |
|
57 |
rouge_score = metric_rouge_score(response, summary)
|
58 |
|
|
|
72 |
|
73 |
def create_batch_evaluation_interface():
|
74 |
with gr.Blocks() as demo:
|
75 |
+
gr.Markdown("## Here are evaluation setups. It will run though datapoints in test_data.josn to generate and evaluate. Show results once finished.")
|
76 |
+
|
77 |
+
model_dropdown = gr.Dropdown(choices=Model.__model_list__, label="Choose a model", value=Model.__model_list__[0])
|
|
|
|
|
78 |
Template_text = gr.Textbox(value="""Summarize the following dialogue""", label='Input Prompting Template', lines=8, placeholder='Input your prompts')
|
79 |
submit_button = gr.Button("✨ Submit ✨")
|
80 |
output = gr.HTML(label="Results")
|
81 |
|
82 |
submit_button.click(
|
83 |
process,
|
84 |
+
inputs=[model_dropdown, Template_text],
|
85 |
outputs=output
|
86 |
)
|
87 |
|
test_samples/test_data.json
ADDED
@@ -0,0 +1 @@
|
|
|
|
|
1 |
+
[{"dialogue": "[doctor] hi michelle what's been going on the medical assistant told me that you injured your knee\n[patient] yeah i'm gon na have to keep this my favorite story is that i had a sandra who injured her knee cold dancing so that would immediately came up to mine but let's try skiing so i was skiing in vermont last weekend and i caught my ski when i was coming off the lift and i fell and my knee popped and it's hurt ever since\n[doctor] okay and were you able to ski down the hill or did you have to be taken down by the ski patrol\n[patient] i had to be taken down by the ski patrol i was unable to bear weight\n[doctor] okay is n't that slightly terrifying they had to do the same thing for me and you're just kind of sitting there watching all of the trees go by so anyway so what part of your knee is bothering you the most is it the inside the outside\n[patient] it hurts on the outside but it also feels like it wo n't hold my weight\n[doctor] okay so it feels like it's gon na buckle\n[patient] hmmm yes\n[doctor] okay alright and are you able to bend it\n[patient] i was immediately after the injury but not really now\n[doctor] okay and how about straightening it are you able to straighten it at all\n[patient] somewhat but it really hurts when i try to straighten all the way\n[doctor] okay alright now do you have any numbing or tingling in your toes\n[patient] no\n[doctor] okay and can you feel your toes okay\n[patient] i think so\n[doctor] okay alright now are you a pretty active person you said you were skiing do you do what kind of other activities do you do\n[patient] i chase my two four -year-old\n[doctor] okay do you have twins\n[patient] i do\n[doctor] nice that's great\n[patient] yeah\n[doctor] so okay and what's your support like at home who is who is able to help out are you still doing that kind of hobbling around on crutches or\n[patient] yeah now my husband is home with me but he also works a job where he is gone quite a bit so it's just me and the kids\n[doctor] okay alright and did the ski patrol give you crutches i i see you have them here with you okay alright great and are you otherwise generally healthy\n[patient] yes\n[doctor] okay and no high blood pressure diabetes anything like that\n[patient] mm-hmm\n[doctor] no okay alright and well i i wan na go ahead let me just do a quick physical exam i'm gon na be calling out some of my exam findings and i'll let you know what that means when i'm done okay alright so on your heart exam your heart sounds in a nice regular rate and rhythm i do n't appreciate any murmur on your lung exam your lungs are nice and clear to auscultation and remind me what knee did you injure again\n[patient] my right knee\n[doctor] your right knee okay so on your right knee examination i do appreciate some edema and an effusion over the right knee does it hurt when i press\n[patient] yes\n[doctor] okay there is pain to palpation of the right lateral knee there is decreased flexion and extension there is a positive lachman sign there is a palpable palpable dorsalis pedis and posterior tibial pulse there is no leg edema in the ankle okay well let's just talk a little bit about you know my assessment and you know my plan for you so you know i know that you had the x-ray done of your of your right knee that did n't show any bony abnormality but i i'm concerned that you have ruptured your your acl or your anterior cruciate ligament that's like a major ligament that helps connect and helps your knee move back and forth so i wan na go ahead and order a knee mri just so that we can get a a a good look and just you know confirm that physical exam okay now some people can have a normal physical exam and their acl can still be torn but you do have a lot of pain on the lateral aspect of your knee so i wan na make sure if there make sure that there is not any other structures that have been damaged by this accident okay i wan na go ahead and you know are you what are you taking for the pain\n[patient] ibuprofen\n[doctor] is that helping\n[patient] somewhat yes\n[doctor] okay alright do you want something stronger\n[patient] no i'm okay\n[doctor] alright so let's just continue with ibuprofen you can take you know six hundred to eight hundred milligrams every eight hours as needed i wan na go ahead and put you in a brace that's gon na help your knee feel a bit more supported okay and let's go ahead and i'm hoping that we can get this mri done in you know the next couple days and then we can have a conversation about what needs to be done now you said that you are are are you know obviously a very active active mom any other exercise or anything else that that i should be aware of that you do just wondering in terms of which kind of graft we would use to fix this with your with your acl being injured\n[patient] hmmm no i not i i would like to get back to the running but currently not that active\n[doctor] okay alright alright do you have any questions about anything\n[patient] i think you've explained it well\n[doctor] okay alright so i'll see you again soon okay\n[patient] okay thank you\n[doctor] alright take care bye", "summary": "CHIEF COMPLAINT\n\nRight knee injury\n\nHISTORY OF PRESENT ILLNESS\n\nMichelle King is a pleasant 44-year-old male who presents to the clinic today for the evaluation of a right knee injury. Her injury was sustained while skiing in Vermont last weekend after she caught her ski when coming off of the lift and fell. She reports a pop in her knee at the time of injury and has been experiencing pain since that time. Due to her fall she was unable to weight bear and had to be taken down by the ski patrol. She locates her pain to the lateral aspect of her knee and also describes the feeling of instability as well as buckling. At this time she is experiencing difficulty with knee flexion; however, she states this was not a problem immediately after the injury. Full knee extension is also quite painful for her. The patient denies any numbness or tingling in her toes. She is currently utilizing crutches for ambulation which were provided to her by the ski patrol. In terms of pain control, she is taking ibuprofen which is providing some relief.\n\nMEDICAL HISTORY\n\nPatient reports she is otherwise healthy and denies a history of high blood pressure or diabetes.\n\nSOCIAL HISTORY\n\nPatient reports she is very active with skiing and taking care of her 4-year-old twins. She mentions that she would like to get back into running at some point. She states that her husband is home with her, but he also works a job where he has gone quite a bit.\n\nMEDICATIONS\n\nPatient reports she is taking ibuprofen.\n\nREVIEW OF SYSTEMS\n\nMusculoskeletal: Reports right knee pain and limited range of motion.\nNeurological: Denies numbness or tingling in toes.\n\nPHYSICAL EXAM\n\nGAIT: The patient is ambulating with crutches.\nCV: Regular rate and rhythm. No murmur.\nRESPIRATORY: Lungs are clear to auscultation.\nMSK: Examination of the right knee: Edema and effusion noted. Pain with palpation of the lateral knee. Decreased flexion and extension. Positive Lachman's. Palpable dorsalis pedis and posterior tibial pulse. No ankle edema.\n\nRESULTS\n\n4 views of the right knee were taken today. These reveal no bony abnormalities.\n\nASSESSMENT\n\nRight knee pain, possible ACL tear.\n\nPLAN\n\nAfter reviewing the patient's examination and radiographic findings today, I have had a lengthy discussion with the patient in regards to her current symptoms. I have recommended that we obtain an MRI of the right knee to evaluate for a possible ACL tear. In the meantime, I have advised her to continue taking ibuprofen 600 to 800 mg every 8 hours as needed for pain. She will also be placed into a brace for increased knee support.\n\nINSTRUCTIONS\n\nThe patient will follow up with me once the MRI results are available for review and further discussion.", "format": "section"}, {"dialogue": "[doctor] okay hi wayne well i understand you're here for you've got a sore on your foot that's not healing is that right\n[patient] yes\n[doctor] so can you tell me about that how are you doing\n[patient] well i've been doing okay but i've had this wound on my right foot for a couple of weeks and it's not getting better i saw my pcp and they referred me to you i i used to see a podiatrist and a couple of years ago but they moved and i was n't able to get another one\n[doctor] okay and how long have you had the wound\n[patient] about two to three weeks\n[doctor] okay have you had any kind of trauma to that foot\n[patient] no i bought a new pair of shoes to travel with and the neuropathy i guess i did n't feel that they were too tight at first it was just a blister but it looked a lot worse now i've been putting a band-aid on it and\n[doctor] okay and so did your doctor put you on any antibiotics when they they saw your foot\n[patient] yes i finished the course yesterday the wound looked about the same though\n[doctor] okay now does it hurt\n[patient] no i ca n't feel it\n[doctor] okay now have you had any other symptoms like fever chills drainage from the wound or anything along those lines\n[patient] no but when i take my sock off sometimes it sticks to the wound even when i put a band-aid on\n[doctor] okay well are you still wearing those same shoes that cause the problem\n[patient] no i'm upset\n[doctor] i'm so upset too i love those shoes\n[patient] no i'm so upset i i love those shoes\n[doctor] okay so what kind of shoes are they\n[patient] they're hoka's\n[doctor] ah so where did you get them\n[patient] i got them at rei couple of years ago\n[doctor] they're pretty good about helping you to fit your fit you in a shoe are n't they\n[patient] yeah\n[doctor] so what do you like about them\n[patient] they are comfortable they are easy to take on and off and they provide good stability\n[doctor] that's good so you've had some issues with stability sometimes or\n[patient] yes\n[doctor] okay okay well another question i want to ask you is how is your diabetes doing\n[patient] i do n't think it's too bad my last hgb a1c was a little over eight\n[doctor] okay alright well let me just do a quick physical exam okay for vital signs your temperature is ninety eight . one your vital signs look good your heart rate is seventy two respirations sixteen blood pressure is one ten over sixty five okay so on your foot exam let's see there is a one by two inch circular wound on the dorsal aspect of the lateral right foot it is just proximal to the right fifth to the fifth mtp joint and there is some yellow slough present with minimal granulation tissue there's no surrounding erythema or cellulitis and there's no evidence of fluid collection there's no necrosis there is no odor i do not appreciate any bony exposure on on vascular exam there are palpable bilateral femoral and popliteal pulses there are no palpable dp or pt pulses but doppler signs are present okay so does this hurt when i touch it here\n[patient] no it's okay\n[doctor] okay alright so i've reviewed the results your right foot x-ray that we did before you i came in the room and this shows no evidence of osteomyelitis which means there is no evidence of bone infection so that's really good so let me tell you a little about my assessment and plan for you so for your first problem your diabetic foot ulcer so i want to order an ankle brachial index abi to determine the blood supply to your foot to see if you can heal this wound i'm also gon na perform a debridement here in the office to take off some of the dead tissue and then next i'm going to prescribe a collagenase ointment to be applied to the wound once daily and then cover with a dry sterile dressing now we will continue this until we see the wound shrinking and a nice pink tissue is present and i want you to wear a surgical shoe to take pressure off of the area okay so i do n't think any more antibiotics are needed at this time and i want to see you again in two weeks so we may need to refer you to a vascular specialist if the abi indicates your blood supply is not optimal for wound healing okay\n[patient] okay\n[doctor] okay so for your diabetes it's gon na be very important to get your diabetes under control in order to get the wound to heal so we might even go ahead and get a diabetic consult so that we can go over some some maybe some ways you can modify your diet without a being too much but maybe that can bring down your hemoglobin a1c into a little bit better level okay\n[patient] okay\n[doctor] alright do you have any other questions\n[patient] no\n[doctor] okay", "summary": "HISTORY OF PRESENT ILLNESS\n\nWayne Jenkins is a pleasant 43-year-old male who presents to the clinic today for the evaluation of a right foot wound, onset 2 to 3 weeks ago. He was seen by his primary care physician who referred him to our office. His last visit with a podiatrist was 2 years ago, however they moved, and he has been unable to get another podiatrist. He denies any specific injury.\n\nThe patient purchased a new pair of shoes and was breaking them in prior to traveling. Initially, he did not feel that the shoes were too tight, however a blister formed on his right foot. The patient does have neuropathy and notes this might be why he did not notice the tightness. His blister soon became a wound and he was seen by his PCP. He was placed on a round of antibiotics, which he completed yesterday. He denies any changes to the appearance of the wound. The patient denies pain or drainage. His sock will occasionally stick to his wound, regardless of if he has a Band-Aid covering it. The patient has discontinued the use of these shoes. Occasionally, he has issues with stability, which was a primary purpose of purchasing these shoes. He denies any fever or chills.\n\nRegarding his diabetes, his last hemoglobin A1C was 8.\n\nREVIEW OF SYSTEMS\n\nConstitutional: Denies fever and chills.\nSkin: Reports right foot wound.\nNeurological: Reports neuropathy.\n\nVITALS\n\nTemperature: 98.1.\nHeart rate: 72.\nRespiration: 16.\nBlood pressure: 110/65 mm Hg.\n\nPHYSICAL EXAM\n\nCV: Palpable bilateral femoral and popliteal pulses. No palpable DP or PT pulses but Doppler signs are present. No evidence of fluid collection.\nSKIN: On the right foot, there is a 1 x 2 inch circular wound on the dorsal aspect of the lateral right foot that is just proximal to the 5th MTP joint. There is some yellow slough present with minimal granulation tissue. No surrounding erythema or cellulitis. No necrosis. No odor.\n\nRESULTS\n\n3 views of the right foot were taken today. These reveal no evidence of osteomyelitis.\n\nASSESSMENT\n\n1. Right foot diabetic ulcer.\n2. Diabetes.\n\nPLAN\n\nAfter reviewing the patient's examination and radiographic findings today, I have had a lengthy discussion with the patient regarding his current symptoms. I have explained to him that his x-rays did not reveal any evidence of osteomyelitis. At this time, I have recommended that we obtain an ankle brachial index ABI to determine the blood supply to his foot to see if he can heal this wound. I have also performed a debridement in the office to remove some of the dead tissue. I have prescribed a collagenase ointment to be applied to the wound 1 time daily and instructed him to cover with a dry sterile dressing. He will continue this until we see the wound shrinking and the nice pink tissue is present. I have also recommended that the patient be placed in a surgical shoe to take pressure off the area. I do not think that antibiotics are needed at this time. He may need to see a vascular specialist if the ABI indicates that his blood supply is not optimal for wound healing.\n\nRegarding his diabetes, I explained the need for controlling his diabetes and the effects this will have on wound healing. I have recommended a diabetic consultation to discuss diet modifications to bring his hemoglobin A1c to a better level.\n\nINSTRUCTIONS\n\nThe patient will follow up with me in 2 weeks to check on his progress.", "format": "section"}, {"dialogue": "Doctor: How old are you young man?\nPatient: I am nineteen. \nDoctor: What happened? How did you get hurt?\nPatient: I was in a fight. Some people are just rude. \nDoctor: Oh, that's bad! How did you hurt your right hand?\nPatient: Ah, I punched that other guy in face or maybe his mouth, I don't remember exactly but he got a good one. Later, in anger I punched in a glass. It broke the glass and spilled all the drink. I think I got a cut as it was bleeding badly. \nDoctor: I see three cuts in your right hand. \nDoctor: When you came here in Emergency your wound was clean and looked like somebody tried to fix your cut. Do you remember any details about it?\nPatient: Yeah, I remember there was a navy corpsman. He was very helpful; he cleaned my cut with the tap water. He also tried to close it up with gauze and superglue. He said it will stop bleeding, but nothing helped. They called for help and here I am in emergency.", "summary": "The patient is a 19-year-old male who was involved in a fight approximately an hour prior to his ED presentation. He punched a guy few times on the face, might be the mouth and then punched a drinking glass, breaking it and lacerating his right hand. He has three lacerations on his right hand. His wound was cleaned out thoroughly with tap water, and one of the navy corpsman tried to use Superglue and gauze to repair it. However, it continues to bleed and he is here for evaluation", "format": "free-form"}, {"dialogue": "Doctor: What medications are you taking currently? \nPatient: Well, I'm taking Remeron for depression and Ziac for high blood pressure.", "summary": "His only medications are Ziac and Remeron.", "format": "free-form"}]
|