# Gemma Project

## Overview
This project involves setting up and running inference using a pre-trained model configured with Low-Rank Adaptation (LoRA). The main components include:
- **gemma.ipynb**: A Jupyter notebook for configuring and experimenting with the model.
- **Inference.py**: A Python script for loading the model and tokenizer, and running inference with specified configurations.

## Files

### gemma.ipynb
This notebook includes:
1. **Loading Lora Configuration**: Setting up the LoRA configuration for the model.
2. **Loading Model and Tokenizer**: Loading the pre-trained model and tokenizer for further tasks.
3. Additional cells likely involve experimenting with model fine-tuning and evaluation.

### Inference.py
This script includes:
1. **Importing Libraries**: Necessary imports including transformers, torch, and specific configurations.
2. **Model and Tokenizer Setup**: Loading the model and tokenizer from the specified paths.
3. **Quantization Configuration**: Applying quantization for efficient model computation.
4. **Inference Execution**: Running inference on the input data.

## Setup

### Requirements
- Python 3.x
- Jupyter Notebook
- PyTorch
- Transformers
- Peft

### Installation
1. Clone the repository:
   ```bash
   git clone <repository_url>
   cd <repository_directory>
   ```
2. Install the required packages:
   ```bash
   pip install torch transformers peft jupyter
   ```

## Usage

### Running the Notebook
1. Open the Jupyter notebook:
   ```bash
   jupyter notebook gemma.ipynb
   ```
2. Follow the instructions in the notebook to configure and experiment with the model.

### Running the Inference Script
1. Execute the inference script:
   ```bash
   python Inference.py
   ```
2. The script will load the model and tokenizer, apply the necessary configurations, and run inference on the provided input.

## Notes
- Ensure that you have the necessary permissions and access tokens for the pre-trained models.
- Adjust the configurations in the notebook and script as needed for your specific use case.

## License
This project is licensed under the MIT License.

## Acknowledgements
- [Hugging Face Transformers](https://huggingface.co/transformers/)
- [PyTorch](https://pytorch.org/)
- [LoRA (Low-Rank Adaptation)](https://arxiv.org/abs/2106.09685)