Ar4l commited on
Commit
89ec711
1 Parent(s): 025cffd

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -21,10 +21,10 @@ Models are named as follows:
21
 
22
  - `CodeBERTa` → `CodeBERTa-ft-coco-[1,2,5]e-05lr`
23
  - e.g. `CodeBERTa-ft-coco-2e-05lr`, which was trained with learning rate of `2e-05`.
24
- - `JonBERTa-head` → `JonBERTa-head-ft-(dense-proj-reinit)`
25
- - e.g. `JonBERTa-head-ft-(dense-proj-)`, where all have `2e-05` learning rate, but may differ in the head layer in which the telemetry features are introduced (either `head` or `proj`).
26
- - `JonBERTa-attn` → `JonBERTa-attn-ft-(0,1,2,3,4,5L)`
27
- - e.g. `JonBERTa-attn-ft-(0,1,2L)` , where all have `2e-05` learning rate, but may differ in the attention layer in which the telemetry features are introduced (either `0`, `1`, `2`, `3`, `4`, or `5L`).
28
 
29
  Other hyperparameters may be found in the paper or the replication package (see below).
30
 
 
21
 
22
  - `CodeBERTa` → `CodeBERTa-ft-coco-[1,2,5]e-05lr`
23
  - e.g. `CodeBERTa-ft-coco-2e-05lr`, which was trained with learning rate of `2e-05`.
24
+ - `JonBERTa-head` → `JonBERTa-head-ft-[dense,proj,reinit]`
25
+ - e.g. `JonBERTa-head-ft-dense-proj`, where all have `2e-05` learning rate, but may differ in the head layer in which the telemetry features are introduced (either `head` or `proj`, with optional `reinit`ialisation of all its weights).
26
+ - `JonBERTa-attn` → `JonBERTa-attn-ft-[0,1,2,3,4,5]L`
27
+ - e.g. `JonBERTa-attn-ft-012L` , where all have `2e-05` learning rate, but may differ in the attention layer(s) in which the telemetry features are introduced (either `0`, `1`, `2`, `3`, `4`, or `5L`).
28
 
29
  Other hyperparameters may be found in the paper or the replication package (see below).
30