Update README.md
Browse files
README.md
CHANGED
@@ -6,11 +6,11 @@ datasets:
|
|
6 |
pipeline_tag: fill-mask
|
7 |
---
|
8 |
|
9 |
-
# Pile of Law BERT large 2
|
10 |
Pretrained model on English language legal and administrative text using the [RoBERTa](https://arxiv.org/abs/1907.11692) pretraining objective. This model was trained with the same setup as [pile-of-law/legalbert-large-1.7M-1](https://huggingface.co/pile-of-law/legalbert-large-1.7M-1), but with a different seed.
|
11 |
|
12 |
## Model description
|
13 |
-
Pile of Law BERT large is a transformers model with the [BERT large model (uncased)](https://huggingface.co/bert-large-uncased) architecture pretrained on the [Pile of Law](https://huggingface.co/datasets/pile-of-law/pile-of-law), a dataset consisting of ~256GB of English language legal and administrative text for language model pretraining.
|
14 |
|
15 |
## Intended uses & limitations
|
16 |
You can use the raw model for masked language modeling or fine-tune it for a downstream task. Since this model was pretrained on a English language legal and administrative text corpus, legal downstream tasks will likely be more in-domain for this model.
|
|
|
6 |
pipeline_tag: fill-mask
|
7 |
---
|
8 |
|
9 |
+
# Pile of Law BERT large model 2 (uncased)
|
10 |
Pretrained model on English language legal and administrative text using the [RoBERTa](https://arxiv.org/abs/1907.11692) pretraining objective. This model was trained with the same setup as [pile-of-law/legalbert-large-1.7M-1](https://huggingface.co/pile-of-law/legalbert-large-1.7M-1), but with a different seed.
|
11 |
|
12 |
## Model description
|
13 |
+
Pile of Law BERT large 2 is a transformers model with the [BERT large model (uncased)](https://huggingface.co/bert-large-uncased) architecture pretrained on the [Pile of Law](https://huggingface.co/datasets/pile-of-law/pile-of-law), a dataset consisting of ~256GB of English language legal and administrative text for language model pretraining.
|
14 |
|
15 |
## Intended uses & limitations
|
16 |
You can use the raw model for masked language modeling or fine-tune it for a downstream task. Since this model was pretrained on a English language legal and administrative text corpus, legal downstream tasks will likely be more in-domain for this model.
|