Hugging Face's logo

uer
/
chinese_roberta_L-2_H-128

Fill-Mask
Transformers
PyTorch
TensorFlow
JAX
Chinese
bert
Inference Endpoints
Model card Files Community
4
uer commited on
Commit
3e0d89c
·
1 Parent(s): a9f8fbb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -95,8 +95,8 @@ python3 preprocess.py --corpus_path corpora/cluecorpus.txt \
95
  ```
96
  python3 pretrain.py --dataset_path cluecorpus_seq128_dataset.pt \
97
  --vocab_path models/google_zh_vocab.txt \
98
- --config_path models/bert_L-x_H-xxx_config.json \
99
- --output_model_path models/cluecorpus_roberta_L-x_H-xxx_seq128_model.bin \
100
  --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \
101
  --total_steps 1000000 --save_checkpoint_steps 100000 --report_steps 50000 \
102
  --learning_rate 1e-4 --batch_size 64 \
@@ -111,10 +111,10 @@ python3 preprocess.py --corpus_path corpora/cluecorpus.txt \
111
  ```
112
  ```
113
  python3 pretrain.py --dataset_path cluecorpus_seq512_dataset.pt \
114
- --pretrained_model_path models/cluecorpus_roberta_L-x_H-xxx_seq128_model.bin-1000000 \
115
  --vocab_path models/google_zh_vocab.txt \
116
- --config_path models/bert_L-x_H-xxx_config.json \
117
- --output_model_path models/cluecorpus_roberta_L-x_H-xxx_seq512_model.bin \
118
  --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \
119
  --total_steps 250000 --save_checkpoint_steps 50000 --report_steps 10000 \
120
  --learning_rate 5e-5 --batch_size 16 \
 
95
  ```
96
  python3 pretrain.py --dataset_path cluecorpus_seq128_dataset.pt \
97
  --vocab_path models/google_zh_vocab.txt \
98
+ --config_path models/bert_tiny_config.json \
99
+ --output_model_path models/cluecorpus_roberta_tiny_seq128_model.bin \
100
  --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \
101
  --total_steps 1000000 --save_checkpoint_steps 100000 --report_steps 50000 \
102
  --learning_rate 1e-4 --batch_size 64 \
 
111
  ```
112
  ```
113
  python3 pretrain.py --dataset_path cluecorpus_seq512_dataset.pt \
114
+ --pretrained_model_path models/cluecorpus_roberta_tiny_seq128_model.bin-1000000 \
115
  --vocab_path models/google_zh_vocab.txt \
116
+ --config_path models/bert_tiny_config.json \
117
+ --output_model_path models/cluecorpus_roberta_tiny_seq512_model.bin \
118
  --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \
119
  --total_steps 250000 --save_checkpoint_steps 50000 --report_steps 10000 \
120
  --learning_rate 5e-5 --batch_size 16 \