Update README.md
Browse files
README.md
CHANGED
|
@@ -20,13 +20,8 @@ Though the training data of this model is CC0, the model itself is not. The mode
|
|
| 20 |
|
| 21 |
## Planned updates
|
| 22 |
As mentioned, a few updates are planned:
|
| 23 |
-
* Further training on more CC0 data, this model's weights will be updated as we pretrain on more of the listed datasets.
|
| 24 |
* Experiment with extending the context length using YaRN to 32k tokens.
|
| 25 |
* Fine-tuning the resulting model for instruct, code and storywriting. These will then be combined using MergeKit to create a MoE model.
|
| 26 |
-
* Release a GGUF version and an extended context version of the base model
|
| 27 |
-
|
| 28 |
-
## Other model versions
|
| 29 |
-
* [Bamboo-400M](https://huggingface.co/KoalaAI/Bamboo-400M)
|
| 30 |
|
| 31 |
# Model Performance Tracking
|
| 32 |
|
|
|
|
| 20 |
|
| 21 |
## Planned updates
|
| 22 |
As mentioned, a few updates are planned:
|
|
|
|
| 23 |
* Experiment with extending the context length using YaRN to 32k tokens.
|
| 24 |
* Fine-tuning the resulting model for instruct, code and storywriting. These will then be combined using MergeKit to create a MoE model.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
# Model Performance Tracking
|
| 27 |
|