TeeZee commited on
Commit
6e08501
1 Parent(s): a34b777

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -4
README.md CHANGED
@@ -114,19 +114,35 @@ model-index:
114
  url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/NEBULA-23B-v1.0
115
  name: Open LLM Leaderboard
116
  ---
117
- ### TeeZee/NEBULA-23B-v1.0 ###
118
 
119
  ![image/png](https://huggingface.co/TeeZee/NEBULA-23B-v1.0/resolve/main/NEBULA-23B-v1.0.jpg)
120
 
121
  - 108 layers,DUS procedure, mistral(32)->SOLAR(48)->GALAXY(72)->NEBULA(108)
 
122
  - model created as a extension of depth upscaling procedure used for SOLAR by upstage
123
 
124
- ### Results
125
  - model can and will produce NSFW content
126
- - for evaluation of RP and ERP more tests are needed
 
 
 
 
127
 
128
- ### Prompt template
 
 
 
 
 
 
129
  - Alpaca
 
 
 
 
 
130
  # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
131
  Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_TeeZee__NEBULA-23B-v1.0)
132
 
 
114
  url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/NEBULA-23B-v1.0
115
  name: Open LLM Leaderboard
116
  ---
117
+ # NEBULA-23.8B-v1.0
118
 
119
  ![image/png](https://huggingface.co/TeeZee/NEBULA-23B-v1.0/resolve/main/NEBULA-23B-v1.0.jpg)
120
 
121
  - 108 layers,DUS procedure, mistral(32)->SOLAR(48)->GALAXY(72)->NEBULA(108)
122
+ - 23.8B parameters
123
  - model created as a extension of depth upscaling procedure used for SOLAR by upstage
124
 
125
+ ## Results
126
  - model can and will produce NSFW content
127
+ - GSM8k evaluation seems to be often broken, HellaSwag, Winograde and TQA show that its a smart model
128
+ - RP and ERP work suprizingly good and I didn't encounter any GPTismsm yet
129
+ - lower memory footprint than 20B and 23B models
130
+ - follows character card very well
131
+ - NSFW output feels fresh comparing to existing models
132
 
133
+ ## Finetuning for RP
134
+ - SFT using MinervaAI/Aesir-Preview dataset, 10 epochs
135
+ - DPO using athirdpath/DPO_Pairs-Roleplay-Alpaca-NSFW dataset, 1 epoch
136
+ - SFT using 1xAda6000, DPO 1x3090, around 24h for each
137
+ - jupyter notebooks or mergekit configs for anyone wanting to reproduce/reuse scripts - just drop me a message
138
+
139
+ ## Prompt template
140
  - Alpaca
141
+ - chat template is embedded in tokenizer config, should load automatically
142
+
143
+ All comments are greatly appreciated, download, test and if you appreciate my work, consider buying me my fuel:
144
+ <a href="https://www.buymeacoffee.com/TeeZee" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 60px !important;width: 217px !important;" ></a>
145
+
146
  # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
147
  Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_TeeZee__NEBULA-23B-v1.0)
148