Ar4l commited on
Commit
99a2247
·
verified ·
1 Parent(s): 78c77b8

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +9 -7
README.md CHANGED
@@ -5,7 +5,7 @@ tags:
5
  - code
6
  ---
7
 
8
- ## CodeBERTa
9
 
10
  Model for the paper [**"A Transformer-Based Approach for Smart Invocation of Automatic Code Completion"**](https://arxiv.org/abs/2405.14753).
11
 
@@ -19,12 +19,14 @@ This model is fine-tuned on a code-completion dataset collected from the open-so
19
 
20
  Models are named as follows:
21
 
22
- - `CodeBERTa` → `CodeBERTa-ft-coco-[1,2,5]e-05lr--[0-4]`
23
- - e.g. `CodeBERTa-ft-coco-1e-05lr--0`, where the trailing `--0` indicates which of the 5 training splits was used.
24
- - `JonBERTa-head` → `JonBERTa-head-ft-(dense-proj-reinit)--[0-4]`
25
- - e.g. `JonBERTa-head-ft-(dense-proj-)--1`, where all have `2e-05` learning rate, but may differ in the head layer in which the telemetry features are introduced (either `head` or `proj`).
26
- - `JonBERTa-attn` → `JonBERTa-attn-ft-(0,1,2,3,4,5L)--[0-4]`
27
- - e.g. `JonBERTa-attn-ft-(0,1,2L)--0` , where all have `2e-05` learning rate, but may differ in the attention layer in which the telemetry features are introduced (either `0`, `1`, `2`, `3`, `4`, or `5L`).
 
 
28
 
29
  #### Sources
30
 
 
5
  - code
6
  ---
7
 
8
+ ## CodeBERTa-ft-coco-1e-05lr
9
 
10
  Model for the paper [**"A Transformer-Based Approach for Smart Invocation of Automatic Code Completion"**](https://arxiv.org/abs/2405.14753).
11
 
 
19
 
20
  Models are named as follows:
21
 
22
+ - `CodeBERTa` → `CodeBERTa-ft-coco-[1,2,5]e-05lr`
23
+ - e.g. `CodeBERTa-ft-coco-2e-05lr`, which was trained with learning rate of `2e-05`.
24
+ - `JonBERTa-head` → `JonBERTa-head-ft-(dense-proj-reinit)`
25
+ - e.g. `JonBERTa-head-ft-(dense-proj-)`, where all have `2e-05` learning rate, but may differ in the head layer in which the telemetry features are introduced (either `head` or `proj`).
26
+ - `JonBERTa-attn` → `JonBERTa-attn-ft-(0,1,2,3,4,5L)`
27
+ - e.g. `JonBERTa-attn-ft-(0,1,2L)` , where all have `2e-05` learning rate, but may differ in the attention layer in which the telemetry features are introduced (either `0`, `1`, `2`, `3`, `4`, or `5L`).
28
+
29
+ Other hyperparameters may be found in the paper or the replication package (see below).
30
 
31
  #### Sources
32