Daniele Licari
commited on
Commit
·
3c27e83
1
Parent(s):
8f704e3
Update README.md
Browse files
README.md
CHANGED
@@ -13,8 +13,19 @@ It achieves better results than the ‘general-purpose’ Italian BERT in differ
|
|
13 |
|
14 |
<h2>Training procedure</h2>
|
15 |
We initialized ITALIAN-LEGAL-BERT with ITALIAN XXL BERT
|
16 |
-
and pretrained for an additional 4 epochs on 3.7 GB of text from the National Jurisprudential
|
17 |
Archive using the Huggingface PyTorch-Transformers library. We used BERT architecture
|
18 |
with a language modeling head on top, AdamW Optimizer, initial learning rate 5e-5 (with
|
19 |
linear learning rate decay, ends at 2.525e-9), sequence length 512, batch size 10 (imposed
|
20 |
-
by GPU capacity), 8.4 million training steps, device 1*GPU V100 16GB
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
13 |
|
14 |
<h2>Training procedure</h2>
|
15 |
We initialized ITALIAN-LEGAL-BERT with ITALIAN XXL BERT
|
16 |
+
and pretrained for an additional 4 epochs on 3.7 GB of preprocessed text from the National Jurisprudential
|
17 |
Archive using the Huggingface PyTorch-Transformers library. We used BERT architecture
|
18 |
with a language modeling head on top, AdamW Optimizer, initial learning rate 5e-5 (with
|
19 |
linear learning rate decay, ends at 2.525e-9), sequence length 512, batch size 10 (imposed
|
20 |
+
by GPU capacity), 8.4 million training steps, device 1*GPU V100 16GB
|
21 |
+
|
22 |
+
## Usage
|
23 |
+
|
24 |
+
ITALIAN-LEGAL-BERT model can be loaded like:
|
25 |
+
|
26 |
+
```python
|
27 |
+
from transformers import AutoModel, AutoTokenizer
|
28 |
+
model_name = "dlicari/Italian-Legal-BERT"
|
29 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
30 |
+
model = AutoModel.from_pretrained(model_name)
|
31 |
+
```
|