|
--- |
|
license: cc-by-nc-4.0 |
|
datasets: |
|
- tatsu-lab/alpaca |
|
language: |
|
- en |
|
|
|
--- |
|
# Eluwa: A Conversational LoRA for Facebook's OPT 2.7b Architecture |
|
|
|
 |
|
|
|
Eluwa is a fine-tuned Low-Rank Adapter (LoRA) model for Facebook's OPT 2.7b. It is trained on the Stanford Alpaca dataset. |
|
The idea was that OPT 2.7 was too curt (and frankly, a bit of an asshole) for a model of its size, and that we could finetune it like Alpaca did to Llama. |
|
|
|
This repository contains the Eluwa 2.7b 2 epoch model, which represents a significant improvements in question-answering ability compared to the default OPT 2.7b model. |
|
Despite Eluwa being not too different from OPT 2.7b, it often does a lot better than default OPT2.7b at accuracy and coherency. |
|
|
|
For example, when asked "What is the capital of Sri Lanka", OPT 2.7b generates nonsense: |
|
``` |
|
A: Sri Lanka is the country where the sun rises every morning on a beautiful beach. It has many beaches, but only one city - Colombo. (The sun sets at night.) |
|
``` |
|
While Eluwa correctly says, |
|
``` |
|
A: Colombo. |
|
``` |
|
|
|
Likewise, when asked how to become a data scientist, Eluwa tries to be useful, whereas OPT 2.7B ends up insulting the user. |
|
Below are the results of Vicuna-style testing: 80 questions in various categories, with the responses rated by GPT-4. |
|
|
|
| Model | OPT 2.7b base | Eluwa 2.7b 1000 iter | Eluwa 2.7b 2 epoch | |
|
|----------------|---------------|----------------------|--------------------| |
|
| Generic | 22 | 44 | 57 | |
|
| Knowledge | 35 | 60 | 72 | |
|
| Roleplay | 29 | 38 | 58 | |
|
| Common sense | 20 | 48 | 50 | |
|
| Fermi | 4 | 28 | 23 | |
|
| Counterfactual | 5 | 24 | 23 | |
|
| Coding | 2 | 7 | 7 | |
|
| Math | 0 | 3 | 3 | |
|
| Writing | 8 | 19 | 19 | |
|
| Total | 125 | 271 | 312 | |
|
|
|
(A sheet of questions, answers and GPT's reviews are also included in this repo). |
|
|
|
Because of its small size, Eluwa can be used as research into conversational models with older and slower hardware. To load it in a UI like oobabooga, |
|
download the model's .bin and .json files, put them in a folder inside the /loras folder, and load it with the OPT 2.7b model. |
|
|
|
|
|
|
