English
Eluwa-2.7b / README.md
Yudhanjaya's picture
Update README.md
dfe2405
|
raw
history blame
1.87 kB
metadata
license: cc-by-nc-4.0
datasets:
  - tatsu-lab/alpaca
language:
  - en

Eluwa: A Conversational LoRA for Facebook's OPT 2.7b Architecture

logo

Eluwa is a fine-tuned Low-Rank Adapter (LoRA) model for Facebook's OPT 2.7b. It is trained on the Stanford Alpaca dataset. The idea was that OPT 2.7 was too curt (and frankly, a bit of an asshole) for a model of its size, and that we could finetune it like Alpaca did to Llama.

This repository contains the Eluwa 2.7b 2 epoch model, which represents a significant improvements in question-answering ability compared to the default OPT 2.7b model. Below are the results of Vicuna-style testing: 80 questions in various categories, with the responses rated by GPT-4.

Model OPT 2.7b base Eluwa 2.7b 1000 iter Eluwa 2.7b 2 epoch
Generic 22 44 57
Knowledge 35 60 72
Roleplay 29 38 58
Common sense 20 48 50
Fermi 4 28 23
Counterfactual 5 24 23
Coding 2 7 7
Math 0 3 3
Writing 8 19 19
Total 125 271 312

Response times are fast: on my GTX 1080ti + Ryzen 3600,it generates between 1.14 tokens/s and 3.77 tokens/s.