Daniele Licari commited on
Commit
3c27e83
·
1 Parent(s): 8f704e3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -2
README.md CHANGED
@@ -13,8 +13,19 @@ It achieves better results than the ‘general-purpose’ Italian BERT in differ
13
 
14
  <h2>Training procedure</h2>
15
  We initialized ITALIAN-LEGAL-BERT with ITALIAN XXL BERT
16
- and pretrained for an additional 4 epochs on 3.7 GB of text from the National Jurisprudential
17
  Archive using the Huggingface PyTorch-Transformers library. We used BERT architecture
18
  with a language modeling head on top, AdamW Optimizer, initial learning rate 5e-5 (with
19
  linear learning rate decay, ends at 2.525e-9), sequence length 512, batch size 10 (imposed
20
- by GPU capacity), 8.4 million training steps, device 1*GPU V100 16GB
 
 
 
 
 
 
 
 
 
 
 
 
13
 
14
  <h2>Training procedure</h2>
15
  We initialized ITALIAN-LEGAL-BERT with ITALIAN XXL BERT
16
+ and pretrained for an additional 4 epochs on 3.7 GB of preprocessed text from the National Jurisprudential
17
  Archive using the Huggingface PyTorch-Transformers library. We used BERT architecture
18
  with a language modeling head on top, AdamW Optimizer, initial learning rate 5e-5 (with
19
  linear learning rate decay, ends at 2.525e-9), sequence length 512, batch size 10 (imposed
20
+ by GPU capacity), 8.4 million training steps, device 1*GPU V100 16GB
21
+
22
+ ## Usage
23
+
24
+ ITALIAN-LEGAL-BERT model can be loaded like:
25
+
26
+ ```python
27
+ from transformers import AutoModel, AutoTokenizer
28
+ model_name = "dlicari/Italian-Legal-BERT"
29
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
30
+ model = AutoModel.from_pretrained(model_name)
31
+ ```