|
--- |
|
license: apache-2.0 |
|
language: |
|
- zh |
|
library_name: transformers |
|
pipeline_tag: text-generation |
|
tags: |
|
- Doctor_consultation |
|
- Taiwan |
|
- fine-tuning |
|
- medicine |
|
--- |
|
|
|
# 🔎 Taiwan-inquiry_7B_v_2.1 |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
"The model was fine-tuned based on the Breeze-7B-Instruct-v1_0 model using a dataset that includes 614 authentic dialogues from the National Cheng Kung University Hospital. |
|
Additionally, 336 synthetic dialogues were included in the training set. The dialogue content covers topics referenced from OSCE (臨床技能測驗) sample questions in Taiwan. These synthetic dialogues were generated using GPT-3.5,Geminio-Pro and Breexe-8x7B-Instruct-v0_1. |
|
The training process was geared towards simulating verbal exchanges between doctors and patients within a hospital environment." |
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
- **Developed by:** [Joseph (Chen-Wei) Li](https://www.linkedin.com/in/joseph-li-3a453b231/), researcher assistant from National Taiwan University Hospital. |
|
- **Model type:** A 7B parameter GPT-like model fine-tuned on a combination of private and synthetic dialogue datasets. |
|
- **Language(s) (NLP):** Traditional Chinese (zh-tw) |
|
- **Finetuned from model :** [Breeze-7B-Instruct-v1_0 ](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v1_0) |
|
|
|
### Usage of the model |
|
- The user can take on the role of a doctor, and the model can engage in conversation with you as if it were a patient. |
|
- You can provide the model with a brief patient background in the system prompt, and the model will respond based on that prompt. (see [Examples](http://www.tame.org.tw/webmag/news/newsFile/508/3.110%E5%B9%B4%E7%AC%AC%E4%B8%80%E6%AC%A1OSCE%20SP%E5%8A%87%E6%83%85%E6%91%98%E8%A6%81%E9%A1%8C%E5%9E%8B%E7%AF%84%E4%BE%8B(63%E4%BE%8B).pdf)) |
|
- Directly asking the certain disease about the symptoms and the possible therapies.**(Warning: It's not medical advice!)** |
|
|
|
### Model evaluation |
|
The model got the **TMMLU+** (0 shot) performance using [EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) with default settings. |
|
|
|
|Details on TMMLU+ (0 shot):<br/>Model | Base Model | STEM | Social Science | Humanities | Other | AVG | |
|
|-----------------------------------------------------|:---------------------:|:---------------:|:--------------:|:----------:|:----------:|:-------:| |
|
| Taiwan-inquiry_7B_v2.1 |Breeze-7B-Instruct-v1_0| 36.46 | 43.94 | 35.68 | 38.21 | 39.38 | |
|
| Taiwan-inquiry_7B_v2.0 |Breeze-7B-Instruct-v0_1 | 36.17 | 43.59 | 35.45 |37.63 | 38.95 | |
|
| Taiwan-inquiry_7B_v1.0 |Taiwan-LLM-7B-v2.1-chat| - | - | - | - | - | |