Update README.md

README.md

@@ -1,5 +1,112 @@
----
-metrics:
-- rouge
-- meteor
----
+# parseny/TinyLlama1.1B-Nvidia-QA
+
+This repository contains the parseny/TinyLlama1.1B-Nvidia-QA model, a fine-tuned version of the TinyLlama language model designed for generating answers on NVIDIA documentation. The model was fine-tuned on a [dataset of question-answer pairs](https://www.kaggle.com/datasets/gondimalladeepesh/nvidia-documentation-question-and-answer-pairs) and evaluated using several metrics to ensure high performance.
+
+## Model Details
+
+- **Model ID**: parseny/TinyLlama1.1B-Nvidia-QA
+- **Model Type**: Causal Language Model
+- **Base Model**: TinyLlama-1.1B
+- **Quantization**: 4-bit quantization using BitsAndBytes
+- **Fine-Tuning Framework**: Hugging Face Transformers and PEFT
+
+## Training Configuration
+
+The model was fine-tuned with the following training arguments:
+
+```python
+training_arguments = TrainingArguments(
+ output_dir="./logs",
+ per_device_train_batch_size=16,
+ gradient_accumulation_steps=4,
+ optim="paged_adamw_32bit",
+ fp16=True,
+ evaluation_strategy="epoch",
+ save_strategy="epoch",
+ num_train_epochs=5,
+ load_best_model_at_end=True,
+ learning_rate=5e-4
+)
+```
+
+## Evaluation Metrics
+
+The performance of the fine-tuned model was evaluated using the following metrics:
+
+- **ROUGE Scores**:
+ - **ROUGE-1**: 0.3122
+ - **ROUGE-2**: 0.1228
+ - **ROUGE-L**: 0.2599
+ - **ROUGE-Lsum**: 0.2600
+
+- **METEOR Score**: 0.27
+
+These scores indicate that the model performs reasonably well in generating responses that are lexically and semantically similar to the reference answers.
+
+## Model Usage
+
+You can use this model to generate responses for chat-based applications. Below is an example of how to load and use the model for generating responses:
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from peft import PeftModel, PeftConfig
+import torch
+
+# Load the model and tokenizer
+model_id = "parseny/TinyLlama1.1B-Nvidia-QA"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id)
+model.to('cuda')
+
+# Generate a response
+generation_config = GenerationConfig(
+ penalty_alpha=0.6, do_sample=True,
+ top_k=5, temperature=0.5, repetition_penalty=1.2,
+ max_new_tokens=47, pad_token_id=tokenizer.eos_token_id
+)
+
+def generate_response(prompt):
+ try:
+ inputs = tokenizer(prompt, return_tensors="pt").to('cuda')
+ outputs = model.generate(**inputs, generation_config=generation_config)
+ generated_response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+ start_idx = generated_response.find('<|im_start|>assistant\n') + len('<|im_start|>assistant\n')
+ generated_response = generated_response[start_idx:]
+ end_idx = generated_response.find('<|im_end|>')
+ generated_response = generated_response[:end_idx]
+ return generated_response
+ except:
+ return ""
+
+# Example usage
+prompt = "What was the purpose of setting up the DGX RAID memory in version 2 of the pipeline?"
+response = generate_response(prompt)
+print(response)
+```
+
+## Training Procedure
+
+The model was fine-tuned using a dataset of question-answer pairs. The fine-tuning process involved:
+
+1. Loading the pre-trained TinyLlama-1.1B model.
+2. Quantizing the model to 4-bit precision to reduce memory usage and increase inference speed.
+3. Fine-tuning the model using the `SFTTrainer` with the specified training arguments.
+4. Evaluating the model at the end of each epoch and saving the best-performing model.
+
+## How to Cite
+
+If you use this model in your research or applications, please cite it as follows:
+
+```
+@misc{parseny-tinyllama-nvidia-qa,
+ author = {Your Name},
+ title = {TinyLlama1.1B-Nvidia-QA: NVIDIA documnetation helper},
+ year = {2024},
+ publisher = {Hugging Face},
+ url = {https://huggingface.co/parseny/TinyLlama1.1B-Nvidia-QA},
+}
+```
+
+## Contact
+
+For any questions or issues, please open an issue on the Hugging Face model repository.