|
--- |
|
license: apache-2.0 |
|
pipeline_tag: text-generation |
|
language: |
|
- en |
|
widget: |
|
- example_title: Hello |
|
messages: |
|
- role: user |
|
content: Hey my name is Julien! How are you? |
|
- example_title: Winter holidays |
|
messages: |
|
- role: system |
|
content: >- |
|
You are a helpful and honest assistant. Please, respond concisely and |
|
truthfully. |
|
- role: user |
|
content: Can you recommend a good destination for Winter holidays? |
|
- example_title: Programming assistant |
|
messages: |
|
- role: system |
|
content: >- |
|
You are a helpful and honest code and programming assistant. Please, |
|
respond concisely and truthfully. |
|
- role: user |
|
content: Write a function that computes the nth fibonacci number. |
|
inference: |
|
parameters: |
|
max_new_tokens: 100 |
|
stop: |
|
- <|end_of_text|> |
|
- <|eot_id|> |
|
--- |
|
WORK IN PROGESS |
|
--- |
|
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr| |
|
|-------------|------:|------|-----:|--------|---|-----:|---|-----:| |
|
|arc_challenge| 1|none | 0|acc |↑ |0.2193|± |0.0121| |
|
| | |none | 0|acc_norm|↑ |0.2517|± |0.0127| |
|
|arc_easy | 1|none | 0|acc |↑ |0.2399|± |0.0088| |
|
| | |none | 0|acc_norm|↑ |0.2428|± |0.0088| |
|
|boolq | 2|none | 0|acc |↑ |0.6116|± |0.0085| |
|
|hellaswag | 1|none | 0|acc |↑ |0.2546|± |0.0043| |
|
| | |none | 0|acc_norm|↑ |0.2647|± |0.0044| |
|
|openbookqa | 1|none | 0|acc |↑ |0.1540|± |0.0162| |
|
| | |none | 0|acc_norm|↑ |0.2680|± |0.0198| |
|
|piqa | 1|none | 0|acc |↑ |0.5413|± |0.0116| |
|
| | |none | 0|acc_norm|↑ |0.5310|± |0.0116| |
|
|winogrande | 1|none | 0|acc |↑ |0.5020|± |0.0141| |
