Cyrile commited on
Commit
0eed571
·
verified ·
1 Parent(s): 479f348

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -10,10 +10,10 @@ refer to the model's page.
10
  ## Impact on performance
11
  Impact of quantization on a set of models.
12
 
13
- Evaluation of the model was conducted using the **PoLL (Pool of LLM)** technique, focusing on **100 French programming and computer science questions** spanning
14
- topics in pseudo-code, Java, Python, and JavaScript. Performance was assessed through scores aggregated from **six evaluations** (two per evaluator). The evaluators
15
- included GPT-4o, Gemini-1.5-pro, and Claude3.5-sonnet, each analyzing the model’s understanding and response accuracy on code syntax, logic, and technical
16
- explanations across these languages and formats.
17
 
18
  Performance Scores (on a scale of 5):
19
  | Model | Score | # params (Billion) | size (GB) |
 
10
  ## Impact on performance
11
  Impact of quantization on a set of models.
12
 
13
+ We evaluated the models using the **PoLL (Pool of LLM)** technique a panel of giga-models (GPT-4o, Gemini Pro 1.5, and Claude-Sonnet 3.5). The scoring ranged from 0,
14
+ indicating a model unsuitable for the task, to 5, representing a model that fully met expectations. The evaluation was based on 67 instructions across four programming
15
+ languages: Python, Java, JavaScript, and Pseudo-code. All tests were conducted in a French-language context, and models were heavily penalized if they responded in
16
+ another language, even if the response was technically correct.
17
 
18
  Performance Scores (on a scale of 5):
19
  | Model | Score | # params (Billion) | size (GB) |