ksyang commited on
Commit
4e12e52
1 Parent(s): 424f69c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -4
README.md CHANGED
@@ -9,9 +9,8 @@ tags:
9
  # **KoBigBird-RoBERTa-large**
10
 
11
  This is a large-sized Korean BigBird model introduced in our [paper]() (IJCNLP-AACL 2023).
12
- The model draws heavily from the parameters of [klue/roberta-large](https://huggingface.co/klue/roberta-large) to ensure high performance
13
- and employs the BigBird architecture to extend its input length.
14
- With the assistance of TAPER to extend position embeddings, the language model's extrapolation capabilities are enhanced.
15
 
16
  ### How to Use
17
 
@@ -33,7 +32,7 @@ Measurement on validation sets of the KLUE benchmark datasets
33
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/62ce3886a9be5c195564fd71/50jMYggkGVUM06n2v1Hxm.png)
34
 
35
  ### Limitations
36
- While our model achieves great results without further pretraining, direct pretraining can further refine positional representations.
37
 
38
  ## Citation Information
39
 
 
9
  # **KoBigBird-RoBERTa-large**
10
 
11
  This is a large-sized Korean BigBird model introduced in our [paper]() (IJCNLP-AACL 2023).
12
+ The model draws heavily from the parameters of [klue/roberta-large](https://huggingface.co/klue/roberta-large) to ensure high performance.
13
+ By employing the BigBird architecture and incorporating the newly proposed TAPER, the language model accommodates even longer input lengths.
 
14
 
15
  ### How to Use
16
 
 
32
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/62ce3886a9be5c195564fd71/50jMYggkGVUM06n2v1Hxm.png)
33
 
34
  ### Limitations
35
+ While our model achieves great results even without additional pretraining, direct pretraining can further refine positional representations.
36
 
37
  ## Citation Information
38