---
license: apache-2.0
pipeline_tag: text-generation
language:
- en
widget:
  - example_title: Hello
    messages:
      - role: user
        content: Hey my name is Julien! How are you?
  - example_title: Winter holidays
    messages:
      - role: system
        content: >-
          You are a helpful and honest assistant. Please, respond concisely and
          truthfully.
      - role: user
        content: Can you recommend a good destination for Winter holidays?
  - example_title: Programming assistant
    messages:
      - role: system
        content: >-
          You are a helpful and honest code and programming assistant. Please,
          respond concisely and truthfully.
      - role: user
        content: Write a function that computes the nth fibonacci number.
inference:
  parameters:
    max_new_tokens: 100
    stop:
      - <|end_of_text|>
      - <|eot_id|>
---
WORK IN PROGESS
---
"|    Tasks    |Version|Filter|n-shot| Metric |   |Value |   |Stderr|
|-------------|------:|------|-----:|--------|---|-----:|---|-----:|
|arc_challenge|      1|none  |     0|acc     |↑  |0.2193|±  |0.0121|
|             |       |none  |     0|acc_norm|↑  |0.2517|±  |0.0127|
|arc_easy     |      1|none  |     0|acc     |↑  |0.2399|±  |0.0088|
|             |       |none  |     0|acc_norm|↑  |0.2428|±  |0.0088|
|boolq        |      2|none  |     0|acc     |↑  |0.6116|±  |0.0085|
|hellaswag    |      1|none  |     0|acc     |↑  |0.2546|±  |0.0043|
|             |       |none  |     0|acc_norm|↑  |0.2647|±  |0.0044|
|openbookqa   |      1|none  |     0|acc     |↑  |0.1540|±  |0.0162|
|             |       |none  |     0|acc_norm|↑  |0.2680|±  |0.0198|
|piqa         |      1|none  |     0|acc     |↑  |0.5413|±  |0.0116|
|             |       |none  |     0|acc_norm|↑  |0.5310|±  |0.0116|
|winogrande   |      1|none  |     0|acc     |↑  |0.5020|±  |0.0141|"