|
|
--- |
|
|
language: |
|
|
- zh |
|
|
tags: |
|
|
- Question Answering |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- webqa |
|
|
- dureader |
|
|
--- |
|
|
|
|
|
# albert-chinese-large-qa |
|
|
Albert large QA model pretrained from baidu webqa and baidu dureader datasets. |
|
|
|
|
|
## Data source |
|
|
+ baidu webqa 1.0 |
|
|
+ baidu dureader |
|
|
|
|
|
## Traing Method |
|
|
We combined the two datasets together and created a new dataset in squad format, including 705139 samples for training and 69638 samples for validation. |
|
|
We finetune the model based on the albert chinese large model. |
|
|
|
|
|
## Hyperparams |
|
|
+ learning_rate 1e-5 |
|
|
+ max_seq_length 512 |
|
|
+ max_query_length 50 |
|
|
+ max_answer_length 300 |
|
|
+ doc_stride 256 |
|
|
+ num_train_epochs 2 |
|
|
+ warmup_steps 1000 |
|
|
+ per_gpu_train_batch_size 8 |
|
|
+ gradient_accumulation_steps 3 |
|
|
+ n_gpu 2 (Nvidia Tesla P100) |
|
|
|
|
|
## Usage |
|
|
``` |
|
|
from transformers import AutoModelForQuestionAnswering, BertTokenizer |
|
|
|
|
|
model = AutoModelForQuestionAnswering.from_pretrained('wptoux/albert-chinese-large-qa') |
|
|
tokenizer = BertTokenizer.from_pretrained('wptoux/albert-chinese-large-qa') |
|
|
``` |
|
|
***Important: use BertTokenizer*** |
|
|
|
|
|
## MoreInfo |
|
|
Please visit https://github.com/wptoux/albert-chinese-large-webqa for details. |
|
|
|
