README.md

---
license: other
library_name: transformers
tags:
- mergekit
- merge
- not-for-all-audiences
base_model:
- Nitral-AI/Infinitely-Laydiculous-7B
model-index:
- name: Infinite-Laymons-9B
 results:
 - task:
 type: text-generation
 name: Text Generation
 dataset:
 name: AI2 Reasoning Challenge (25-Shot)
 type: ai2_arc
 config: ARC-Challenge
 split: test
 args:
 num_few_shot: 25
 metrics:
 - type: acc_norm
 value: 65.61
 name: normalized accuracy
 source:
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Infinite-Laymons-9B
 name: Open LLM Leaderboard
 - task:
 type: text-generation
 name: Text Generation
 dataset:
 name: HellaSwag (10-Shot)
 type: hellaswag
 split: validation
 args:
 num_few_shot: 10
 metrics:
 - type: acc_norm
 value: 84.14
 name: normalized accuracy
 source:
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Infinite-Laymons-9B
 name: Open LLM Leaderboard
 - task:
 type: text-generation
 name: Text Generation
 dataset:
 name: MMLU (5-Shot)
 type: cais/mmlu
 config: all
 split: test
 args:
 num_few_shot: 5
 metrics:
 - type: acc
 value: 64.53
 name: accuracy
 source:
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Infinite-Laymons-9B
 name: Open LLM Leaderboard
 - task:
 type: text-generation
 name: Text Generation
 dataset:
 name: TruthfulQA (0-shot)
 type: truthful_qa
 config: multiple_choice
 split: validation
 args:
 num_few_shot: 0
 metrics:
 - type: mc2
 value: 54.87
 source:
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Infinite-Laymons-9B
 name: Open LLM Leaderboard
 - task:
 type: text-generation
 name: Text Generation
 dataset:
 name: Winogrande (5-shot)
 type: winogrande
 config: winogrande_xl
 split: validation
 args:
 num_few_shot: 5
 metrics:
 - type: acc
 value: 80.82
 name: accuracy
 source:
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Infinite-Laymons-9B
 name: Open LLM Leaderboard
 - task:
 type: text-generation
 name: Text Generation
 dataset:
 name: GSM8k (5-shot)
 type: gsm8k
 config: main
 split: test
 args:
 num_few_shot: 5
 metrics:
 - type: acc
 value: 53.75
 name: accuracy
 source:
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Infinite-Laymons-9B
 name: Open LLM Leaderboard
---
# GGUF / IQ / Imatrix for [Infinite-Laymons-9B](https://huggingface.co/ABX-AI/Infinite-Laymons-9B)
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65d936ad52eca001fdcd3245/_Tgq278Uqjns2W0ug1kXE.png)

**Why Importance Matrix?**

**Importance Matrix**, at least based on my testing, has shown to improve the output and performance of "IQ"-type quantizations, where the compression becomes quite heavy.
The **Imatrix** performs a calibration, using a provided dataset. Testing has shown that semi-randomized data can help perserve more important segments as the compression is applied.

Related discussions in Github:
[[1]](https://github.com/ggerganov/llama.cpp/discussions/5006) [[2]](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384)

The imatrix.txt file that I used contains general, semi-random data, with some custom kink.

# Infinite-Laymons-9B

Infinite-Laymons-9B is intended for fictional role-play and storytelling. 

The focus is on original responses and elimitation, or reduction of refusals.

## Merge Details

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

### Merge Method

This model was merged using the passthrough merge method.

### Models Merged

The following models were included in the merge:
* [Nitral-AI/Infinitely-Laydiculous-7B](https://huggingface.co/Nitral-AI/Infinitely-Laydiculous-7B)
* [ABX-AI/Infinite-Laymons-7B](https://huggingface.co/ABX-AI/Infinite-Laymons-7B)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
slices:
 - sources:
 - model: Nitral-AI/Infinitely-Laydiculous-7B
 layer_range: [0, 20]
 - sources:
 - model: ABX-AI/Infinite-Laymons-7B
 layer_range: [12, 32]
merge_method: passthrough
dtype: float16
```

# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ABX-AI__Infinite-Laymons-9B)

| Metric |Value|
|---------------------------------|----:|
|Avg. |67.29|
|AI2 Reasoning Challenge (25-Shot)|65.61|
|HellaSwag (10-Shot) |84.14|
|MMLU (5-Shot) |64.53|
|TruthfulQA (0-shot) |54.87|
|Winogrande (5-shot) |80.82|
|GSM8k (5-shot) |53.75|