license: apache-2.0 language: - en metrics: - rouge base_model: - microsoft/phi-2 pipeline_tag: question-answering
Model Card for PEFT-Fine-Tuned Model
This model card documents a PEFT-fine-tuned version of microsoft/phi-2
for question-answering tasks. The PEFT fine-tuning improved the model's performance, as detailed in the evaluation section.
Model Details
Model Description
- Developed by: User (you can replace with your name or organization)
- Finetuned from model:
microsoft/phi-2
- Model type: PEFT fine-tuned transformer
- Language(s) (NLP): English
- License: Apache 2.0
The base model microsoft/phi-2
was adapted using Parameter-Efficient Fine-Tuning (PEFT) for question-answering tasks. The training process focused on improving performance metrics while keeping computational costs low.
Model Sources
- Repository: https://huggingface.co/JamieAi33/Phi-2_PEFT
Uses
Direct Use
This model can be used out-of-the-box for question-answering tasks.
Downstream Use
The model can be fine-tuned further on domain-specific datasets for improved performance.
Out-of-Scope Use
Avoid using this model for tasks outside question-answering or where fairness, bias, and ethical considerations are critical without further validation.
Bias, Risks, and Limitations
Users should be aware that:
- The model is trained on publicly available data and may inherit biases present in the training data.
- It is optimized for English and may perform poorly in other languages.
How to Get Started with the Model
Here’s an example of loading the model:
from transformers import AutoModel
from peft import PeftModel
base_model = AutoModel.from_pretrained("microsoft/phi-2")
adapter_model = PeftModel.from_pretrained(base_model, "Phi-2_PEFT")
# Model Name: PEFT Fine-Tuned `microsoft/phi-2`
This repository contains a PEFT fine-tuned version of the `microsoft/phi-2` model for question-answering tasks. The fine-tuning process leveraged Parameter-Efficient Fine-Tuning (PEFT) techniques to achieve improved performance.
---
## Metrics
The model's performance was evaluated using the ROUGE metric. Below are the results:
| **Metric** | **Original Model** | **PEFT Model** | **Absolute Improvement** |
|-----------------|--------------------|----------------|---------------------------|
| **ROUGE-1** | 29.76% | 44.51% | +14.75% |
| **ROUGE-2** | 10.76% | 15.68% | +4.92% |
| **ROUGE-L** | 21.69% | 30.95% | +9.25% |
| **ROUGE-Lsum** | 22.75% | 31.49% | +8.74% |
---
- Downloads last month
- 6
Model tree for JamieAi33/Phi-2_PEFT
Base model
microsoft/phi-2