README.md

metadata

library_name: peft
base_model: mistralai/Mistral-7B-v0.1
language:
  - en
tags:
  - Δ
  - LoRA

Model Details

$$W_{mistral}+LoR{A}_{hermes}=W_{hermes}{W}_{hermes}-LoR{A}_{hermes}=W_{mistral}W\_\{mistral\}\; +\; LoRA\_\{hermes\}\; =\; W\_\{hermes\}\; \backslash \backslash \; W\_\{hermes\}\; -\; LoRA\_\{hermes\}\; =\; W\_\{mistral\}$$

Why Though?

unfortunately this is not as simple as typeof/zephyr-7b-beta-lora due to the way in which OpenHermes was trained... by adding tokens, the corresponance is not 1-to-1 with mistralai/Mistral-7B-v0.1 as is the case with typeof/zephyr-7b-beta-lora ... nevertheless, if you have found yourself here, I'm sure you can figure out how to use it... if not, open up an issue!

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Summary

A fine-tuned version of mistralai/Mistral-7B-v0.1

LoRA

QLoRA