---
library_name: transformers
tags: []
---

|                    | Commonsense (Micro) | Commonsense (Macro) | Hard (Micro) | Hard (Macro) | Final Pass Rate |
|--------------------|:-------------------:|:-------------------:|:------------:|:------------:|:---------------:|
| **Direct Prompting**|                     |                     |              |              |                 |
| Llama3.1-8B        |        60.1          |         0.0          |      7.9     |      2.8     |       0.0       |
| Qwen2-7B           |        49.9          |         1.1          |      2.1     |      0.0     |       0.0       |
| **Fine-tuning** |           |                     |              |              |                 |
| Llama3.1-8B        |        78.3          |        17.8          |     19.3     |      6.1     |       3.8       |
| Qwen2-7B           |        59.0          |         0.6          |      0.2     |      0.0     |       0.0       |
| **Δ**              |      +12.1           |      +23.7           |    +6.4      |    +2.2      |      +7.8       |