Update README.md
Browse files
README.md
CHANGED
@@ -19,10 +19,10 @@ tags:
|
|
19 |
|
20 |
# Kexer models
|
21 |
|
22 |
-
Kexer models
|
23 |
-
This is a repository for fine-tuned Deepseek-coder-6.7b model in the Hugging Face Transformers format.
|
24 |
|
25 |
-
#
|
26 |
|
27 |
```python
|
28 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
@@ -73,20 +73,19 @@ More details about finetuning can be found in the technical report
|
|
73 |
|
74 |
# Fine-tuning data
|
75 |
|
76 |
-
For this model we used 15K exmaples
|
77 |
-
For more information about the dataset follow the link.
|
78 |
|
79 |
# Evaluation
|
80 |
|
81 |
-
|
82 |
|
83 |
-
|
84 |
|
85 |
| **Model name** | **Kotlin HumanEval Pass Rate** |
|
86 |
|:---------------------------:|:----------------------------------------:|
|
87 |
-
| `
|
88 |
-
| `
|
89 |
|
90 |
# Ethical Considerations and Limitations
|
91 |
|
92 |
-
Deepseek-7B-Kexer
|
|
|
19 |
|
20 |
# Kexer models
|
21 |
|
22 |
+
Kexer models are a collection of open-source generative text models fine-tuned on the [Kotlin Exercices](https://huggingface.co/datasets/JetBrains/KExercises) dataset.
|
23 |
+
This is a repository for the fine-tuned **Deepseek-coder-6.7b** model in the *Hugging Face Transformers* format.
|
24 |
|
25 |
+
# How to use
|
26 |
|
27 |
```python
|
28 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
73 |
|
74 |
# Fine-tuning data
|
75 |
|
76 |
+
For tuning this model, we used 15K exmaples from the synthetically generated [Kotlin Exercices dataset](https://huggingface.co/datasets/JetBrains/KExercises). Every example follows the HumanEval format. In total, the dataset contains about 3.5M tokens.
|
|
|
77 |
|
78 |
# Evaluation
|
79 |
|
80 |
+
For evaluation, we used the [Kotlin HumanEval](https://huggingface.co/datasets/JetBrains/Kotlin_HumanEval) dataset, which contains all 161 tasks from HumanEval translated into Kotlin by human experts. You can find more details about the pre-processing necessary to obtain our results, including the code for running, on the [datasets's page](https://huggingface.co/datasets/JetBrains/Kotlin_HumanEval).
|
81 |
|
82 |
+
Here are the results of our evaluation:
|
83 |
|
84 |
| **Model name** | **Kotlin HumanEval Pass Rate** |
|
85 |
|:---------------------------:|:----------------------------------------:|
|
86 |
+
| `Deepseek-7B` | 40.99 |
|
87 |
+
| `Deepseek-7B-Kexer` | **55.28** |
|
88 |
|
89 |
# Ethical Considerations and Limitations
|
90 |
|
91 |
+
Deepseek-7B-Kexer is a new technology that carries risks with use. The testing conducted to date has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Deepseek-7B-Kexer's potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate or objectionable responses to user prompts. The model was fine-tuned on a specific data format (Kotlin tasks), and deviation from this format can also lead to inaccurate or undesirable responses to user queries. Therefore, before deploying any applications of Deepseek-7B-Kexer, developers should perform safety testing and tuning tailored to their specific applications of the model.
|