Update README.md
Browse files
README.md
CHANGED
@@ -34,19 +34,28 @@ We've modified Flax's 'lm1b' example to train on Japanese dataset. You can find
|
|
34 |
|
35 |
## Usage
|
36 |
|
37 |
-
|
38 |
|
39 |
-
*
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
40 |
|
41 |
```
|
42 |
-
|
43 |
```
|
44 |
|
45 |
Run the Python interpreter and download the model files.
|
46 |
|
47 |
```
|
48 |
-
|
49 |
-
|
50 |
```
|
51 |
|
52 |
```python
|
@@ -58,16 +67,18 @@ Run the Python interpreter and download the model files.
|
|
58 |
Clone the source code and install the necessary Python packages.
|
59 |
|
60 |
```
|
61 |
-
|
62 |
-
|
63 |
-
|
64 |
-
|
|
|
|
|
65 |
```
|
66 |
|
67 |
Text generation using downloaded model files.
|
68 |
|
69 |
```
|
70 |
-
|
71 |
--config=configs/japanese_0.1b_v1.py \
|
72 |
--config.sampling_temperature=0.6 \
|
73 |
--config.sampling_top_k=20 \
|
|
|
34 |
|
35 |
## Usage
|
36 |
|
37 |
+
Here, we explain the procedure to generate text from pretrained weights using a CPU. We used the following instance on GCE for the Python 3.8 environment.
|
38 |
|
39 |
+
* Machine Type: c2-standard-4 (4 CPUs, 16GB Memory)
|
40 |
+
* Disk: 100GB (Standard Persistent Disk)
|
41 |
+
* OS: Ubuntu 20.04 LTS x86/64
|
42 |
+
|
43 |
+
Install Python 3.8 and pip.
|
44 |
+
|
45 |
+
```
|
46 |
+
$ sudo apt-get update
|
47 |
+
$ sudo apt-get install python3.8 python3-pip build-essential
|
48 |
+
```
|
49 |
|
50 |
```
|
51 |
+
$ pip install --upgrade huggingface_hub
|
52 |
```
|
53 |
|
54 |
Run the Python interpreter and download the model files.
|
55 |
|
56 |
```
|
57 |
+
$ cd $HOME
|
58 |
+
$ python3
|
59 |
```
|
60 |
|
61 |
```python
|
|
|
67 |
Clone the source code and install the necessary Python packages.
|
68 |
|
69 |
```
|
70 |
+
$ git clone -b 1.0.0.RC2 https://github.com/FookieMonster/transformer-lm-japanese
|
71 |
+
$ cd ./transformer-lm-japanese/transformer_lm
|
72 |
+
$ pip install -r requirements.txt
|
73 |
+
$ pip install jax[cpu]==0.3.2
|
74 |
+
$ pip install chex==0.1.5
|
75 |
+
$ pip install protobuf==3.20.3
|
76 |
```
|
77 |
|
78 |
Text generation using downloaded model files.
|
79 |
|
80 |
```
|
81 |
+
$ python3 generate_text.py --workdir=$HOME/logs/japanese_0.1b_v1 \
|
82 |
--config=configs/japanese_0.1b_v1.py \
|
83 |
--config.sampling_temperature=0.6 \
|
84 |
--config.sampling_top_k=20 \
|