Update README.md (#3)
Browse files- Update README.md (4cb75db75e271b9bd2dea8da324f4293687bf169)
Co-authored-by: Mikhail Evtikhiev <[email protected]>
README.md
CHANGED
@@ -86,15 +86,15 @@ We clean the content of the remaining dataset entries according to the following
|
|
86 |
|
87 |
# Evaluation
|
88 |
|
89 |
-
To evaluate we used Kotlin Humaneval
|
90 |
|
91 |
Fine-tuned model:
|
92 |
|
93 |
| **Model name** | **Kotlin HumanEval Pass Rate** |
|
94 |
|:---------------------------:|:----------------------------------------:|
|
95 |
| `base model` | 26.09 |
|
96 |
-
| `fine-tuned model` | 29.19 |
|
97 |
|
98 |
# Ethical Considerations and Limitations
|
99 |
|
100 |
-
|
|
|
86 |
|
87 |
# Evaluation
|
88 |
|
89 |
+
To evaluate we used [Kotlin Humaneval](https://huggingface.co/datasets/JetBrains/Kotlin_HumanEval)
|
90 |
|
91 |
Fine-tuned model:
|
92 |
|
93 |
| **Model name** | **Kotlin HumanEval Pass Rate** |
|
94 |
|:---------------------------:|:----------------------------------------:|
|
95 |
| `base model` | 26.09 |
|
96 |
+
| `fine-tuned model` | **29.19** |
|
97 |
|
98 |
# Ethical Considerations and Limitations
|
99 |
|
100 |
+
CodeLlama-7B-KStack-full and its variants are a new technology that carries risks with use. The testing conducted to date could not cover all scenarios. For these reasons, as with all LLMs, CodeLlama-7B-KStack-full's potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate or objectionable responses to user prompts. The model was fine-tuned on a specific data format (Kotlin tasks), and deviation from this format can also lead to inaccurate or undesirable responses to user queries. Therefore, before deploying any applications of CodeLlama-7B-KStack-full, developers should perform safety testing and tuning tailored to their specific applications of the model.
|