File size: 999 Bytes
b5066cf 39e5852 b5066cf 388f3eb |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
---
license: llama2
tags:
- merge
- mergekit
---
Llama 2 13b is a pretty decent language model. You know what's probably better? *Two* Llama 2 13b models. In a trenchcoat.
Produced by [`bakllama.py`](https://github.com/cg123/mergekit/blob/main/bakllama.py) with this config file:
```yml
layer_slices:
- model: TheBloke/Llama-2-13B-fp16
start: 0
end: 40
- model: TheBloke/Llama-2-13B-fp16
start: 0
end: 40
```
No fine tuning was done on this model. Yes, it's still coherent somehow.
Benchmark results:
| Benchmark | Llama2-13b | Llama2-26b-tcs | Percent Change |
| --- | --- | --- | --- |
| ARC | 59.3 | 55.03 | -7.2% |
| HellaSwag | 82.15 | 79.9 | -2.74% |
| MMLU | 55.67 | 53.73| -3.48% |
| TruthfulQA | 37.39 | 40.48 | +5.59% |
| Average | 58.63 | 57.29 | -2.29% |
| Average Minus TQA | 65.70 | 62.85 | -4.34% |
This tells us two very important things:
1. TruthfulQA is a perfect benchmark in every way.
2. Llama models are amazingly robust to being fed their own output. |