File size: 1,915 Bytes
672bf5c
 
 
 
 
38c02b3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
dcd5886
38c02b3
 
dcd5886
61e88e0
 
c5fad0d
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
---
license: apache-2.0
pipeline_tag: text-generation
language:
- en
widget:
  - example_title: Hello
    messages:
      - role: user
        content: Hey my name is Julien! How are you?
  - example_title: Winter holidays
    messages:
      - role: system
        content: >-
          You are a helpful and honest assistant. Please, respond concisely and
          truthfully.
      - role: user
        content: Can you recommend a good destination for Winter holidays?
  - example_title: Programming assistant
    messages:
      - role: system
        content: >-
          You are a helpful and honest code and programming assistant. Please,
          respond concisely and truthfully.
      - role: user
        content: Write a function that computes the nth fibonacci number.
inference:
  parameters:
    max_new_tokens: 100
    stop:
      - <|end_of_text|>
      - <|eot_id|>
---
WORK IN PROGESS
---
"|    Tasks    |Version|Filter|n-shot| Metric |   |Value |   |Stderr|
|-------------|------:|------|-----:|--------|---|-----:|---|-----:|
|arc_challenge|      1|none  |     0|acc     |↑  |0.2193|±  |0.0121|
|             |       |none  |     0|acc_norm|↑  |0.2517|±  |0.0127|
|arc_easy     |      1|none  |     0|acc     |↑  |0.2399|±  |0.0088|
|             |       |none  |     0|acc_norm|↑  |0.2428|±  |0.0088|
|boolq        |      2|none  |     0|acc     |↑  |0.6116|±  |0.0085|
|hellaswag    |      1|none  |     0|acc     |↑  |0.2546|±  |0.0043|
|             |       |none  |     0|acc_norm|↑  |0.2647|±  |0.0044|
|openbookqa   |      1|none  |     0|acc     |↑  |0.1540|±  |0.0162|
|             |       |none  |     0|acc_norm|↑  |0.2680|±  |0.0198|
|piqa         |      1|none  |     0|acc     |↑  |0.5413|±  |0.0116|
|             |       |none  |     0|acc_norm|↑  |0.5310|±  |0.0116|
|winogrande   |      1|none  |     0|acc     |↑  |0.5020|±  |0.0141|"