crumb commited on
Commit
78a2e90
·
1 Parent(s): 8ccae35

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -5
README.md CHANGED
@@ -11,20 +11,16 @@ pinned: false
11
 
12
  Who needs em, we all have em, they're just like us. Unusable models, compute optimally 🔥. We hope that by open-sourcing our compute-optimal trained models, that others can replicate our results and also make no use out of our unusable models. These models are not useful in the slightest, and don't benefit research. Every time you use one of these models, you can be sure that you will not get a useful result, and every time we kiss I swear I can fly. Can't you feel my heart beat fast, I want this to last, need you by my side. We introduce a cascade(a) (sorry) of classes and models.
13
 
14
- Evaluations and more information about the training for every Gerbil model can be found here: https://github.com/aicrumb/notebook-hosting/blob/main/GerbilLabEvaluations.md
15
 
16
  Special tokens for "Blender" models' pretraining include:
17
 
18
  ```
19
  '<fitm_start>', '<multiple_tok_mask>', '<fitm_result>', '<causal>', '<mlm_start>', '<single_tok_mask>', '<mlm_end>'
20
-
21
  # Example fill in the middle
22
  '<fitm_start> this is an <multiple_tok_mask> for fill-in-the-middle <fitm_result> example text <|endoftext|>'
23
-
24
  # Example causal language modelling
25
  '<causal> this is an example text for causal language modelling <|endoftext|>'
26
-
27
  # Example masked language modelling
28
  '<mlm_start> this is an <single_tok_mask> text for masked language modelling <mlm_end> example <|endoftext|>'
29
-
30
  ```
 
11
 
12
  Who needs em, we all have em, they're just like us. Unusable models, compute optimally 🔥. We hope that by open-sourcing our compute-optimal trained models, that others can replicate our results and also make no use out of our unusable models. These models are not useful in the slightest, and don't benefit research. Every time you use one of these models, you can be sure that you will not get a useful result, and every time we kiss I swear I can fly. Can't you feel my heart beat fast, I want this to last, need you by my side. We introduce a cascade(a) (sorry) of classes and models.
13
 
14
+ Evaluations and more information about the training for every Gerbil model and the mixture-of-tasks Blender pretraining method can be found here: https://github.com/aicrumb/notebook-hosting/blob/main/GerbilLabEvaluations.md
15
 
16
  Special tokens for "Blender" models' pretraining include:
17
 
18
  ```
19
  '<fitm_start>', '<multiple_tok_mask>', '<fitm_result>', '<causal>', '<mlm_start>', '<single_tok_mask>', '<mlm_end>'
 
20
  # Example fill in the middle
21
  '<fitm_start> this is an <multiple_tok_mask> for fill-in-the-middle <fitm_result> example text <|endoftext|>'
 
22
  # Example causal language modelling
23
  '<causal> this is an example text for causal language modelling <|endoftext|>'
 
24
  # Example masked language modelling
25
  '<mlm_start> this is an <single_tok_mask> text for masked language modelling <mlm_end> example <|endoftext|>'
 
26
  ```