Update README.md

README.md

@@ -1,3 +1,68 @@
 ---
 license: apache-2.0
+pipeline_tag: text-generation
+tags:
+- llm-foundry
+- docsgpt
 ---
+
+DocsGPT-7B is a decoder-style transformer that is fine-tuned specifically for providing answers based on documentation given in context. It is an extension of the MosaicPretrainedTransformer (MPT) family, being fine-tuned from the MPT-7B model developed by MosaicML. The model inherits the powerful language understanding capabilities of MPT-7B and has been specialized for the purpose of documentation-oriented question answering.
+
+## Model Description
+
+    Architecture: Decoder-style Transformer
+    Language: English
+    Training data: Fine-tuned on approximately 1000 high-quality examples of documentation answering workflows.
+    Base model: Fine-tuned version of MPT-7B, which is pretrained from scratch on 1T tokens of English text and code.
+    License: Apache 2.0
+
+## Features
+
+* Attention with Linear Biases (ALiBi): Inherited from the MPT family, this feature eliminates the context length limits by replacing positional embeddings, allowing for efficient and effective processing of lengthy documents. In future we are planning to finish training on our larger dataset and to increase amount of tokens for context.
+* Optimized for Documentation: Specifically fine-tuned for providing answers that are based on documentation provided in context, making it particularly useful for developers and technical support teams.
+* Easy to Serve: Can be efficiently served using standard HuggingFace pipelines or NVIDIA's FasterTransformer.
+
+
+## How to Use
+
+This model is best used with the MosaicML [llm-foundry repository](https://github.com/mosaicml/llm-foundry) for training and finetuning.
+
+```python
+import transformers
+model = transformers.AutoModelForCausalLM.from_pretrained(
+  'mosaicml/mpt-7b',
+  trust_remote_code=True
+)
+```
+
+
+This model was uses [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) tokenizer.
+
+```python
+from transformers import AutoTokenizer
+tokenizer = AutoTokenizer.from_pretrained('EleutherAI/gpt-neox-20b')
+```
+
+
+## Documentation
+
+* [Base model documentation](https://github.com/mosaicml/llm-foundry/)
+* Our community [Discord](https://discord.gg/n5BX8dh8rU)
+* [DocGPT](https://github.com/arc53/DocsGPT) project
+ 
+
+
+## Disclaimer
+
+The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please cosult an attorney before using this model for commercial purposes.
+
+## Limitations
+
+Please be aware this is a relatively small llm and its prone to biases and hallucinations
+
+
+Our live [demo](https://docsgpt.arc53.com/) that uses a mixture of models
+
+## Model License
+
+Apache-2.0