Initializing audio transcriptor

README.md

@@ -1,13 +1,190 @@
----
-title: Audio Transcriptor
-emoji: 🐢
-colorFrom: gray
-colorTo: red
-sdk: gradio
-sdk_version: 5.3.0
-app_file: app.py
-pinned: false
-short_description: This project involves a Python-based audio transcription
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Audio Transcription and Diarization Tool
+
+## Overview
+
+This project provides a robust set of tools for transcribing audio files using the Whisper model and performing speaker diarization with PyAnnote. Users can process audio files, record audio, and save transcriptions with speaker identification.
+
+## Table of Contents
+- [Features](#features)
+- [Requirements](#requirements)
+- [Setup](#setup)
+- [Usage](#usage)
+  - [Basic Example](#basic-example)
+  - [Audio Processing Example](#audio-processing-example)
+  - [Transcribing an Existing Audio File or Recording](#transcribing-an-existing-audio-file-or-recording)
+- [Key Components](#key-components)
+  - [Transcriptor](#transcriptor)
+  - [AudioProcessor](#audioprocessor)
+  - [AudioRecording](#audiorecording)
+- [Contributing](#contributing)
+- [Acknowledgments](#acknowledgments)
+
+## Features
+
+- **Transcription**: Convert audio files in various formats to text (automatically converts to WAV).
+- **Speaker Diarization**: Identify different speakers in the audio.
+- **Speaker Retrieval**: Name speakers during transcription.
+- **Audio Recording**: Record audio directly from a microphone.
+- **Audio Preprocessing**: Includes resampling, format conversion, and audio enhancement.
+- **Multiple Model Support**: Choose from various Whisper model sizes.
+
+## Supported Whisper Models
+
+This tool supports various Whisper model sizes, allowing you to balance accuracy and computational resources:
+
+- **`tiny`**: Fastest, lowest accuracy
+- **`base`**: Fast, good accuracy
+- **`small`**: Balanced speed and accuracy
+- **`medium`**: High accuracy, slower
+- **`large`**: High accuracy, resource-intensive
+- **`large-v1`**: Improved large model
+- **`large-v2`**: Further improved large model
+- **`large-v3`**: Latest and most accurate
+- **`large-v3-turbo`**: Optimized for faster processing
+
+Specify the model size when initializing the Transcriptor:
+
+```python
+transcriptor = Transcriptor(model_size="base")
+```
+
+The default model size is "base" if not specified.
+
+## Requirements
+
+To run this project, you need Python 3.7+ and the following packages:
+
+```plaintext
+- openai-whisper
+- pyannote.audio
+- librosa
+- tqdm
+- python-dotenv
+- termcolor
+- pydub
+- SpeechRecognition
+- pyaudio
+- tabulate
+- soundfile
+- torch
+- numpy
+- transformers
+- gradio
+```
+
+Install the required packages using:
+
+```bash
+pip install -r requirements.txt
+```
+
+## Setup
+
+1. **Clone the repository**:
+   ```bash
+   git clone https://github.com/your-username/audio-transcription-tool.git
+   cd audio-transcription-tool
+   ```
+
+2. **Install the required packages**:
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+3. **Set up your environment variables**:
+   - Create a `.env` file in the root directory.
+   - Add your Hugging Face token:
+     ```plaintext
+     HF_TOKEN=your_hugging_face_token_here
+     ```
+
+## Usage
+
+### Basic Example
+
+Here's how to use the Transcriptor class to transcribe an audio file:
+
+```python
+from pyscript import Transcriptor
+
+# Initialize the Transcriptor
+transcriptor = Transcriptor()
+
+# Transcribe an audio file
+transcription = transcriptor.transcribe_audio("/path/to/audio")
+
+# Interactively name speakers
+transcription.get_name_speakers()
+
+# Save the transcription
+transcription.save()
+```
+
+### Audio Processing Example
+
+Use the AudioProcessor class to preprocess your audio files:
+
+```python
+from pyscript import AudioProcessor
+
+# Load an audio file
+audio = AudioProcessor("/path/to/audio.mp3")
+
+# Display audio details
+audio.display_details()
+
+# Convert to WAV format and resample to 16000 Hz
+audio.convert_to_wav()
+
+# Display updated audio details
+audio.display_changes()
+```
+
+### Transcribing an Existing Audio File or Recording
+
+To transcribe an audio file or record and transcribe audio, use the demo application provided in `demo.py`:
+
+```bash
+python demo.py
+```
+
+## Key Components
+
+### Transcriptor
+
+The `Transcriptor` class (in `pyscript/transcriptor.py`) is the core of the transcription process. It handles:
+
+- Loading the Whisper model
+- Setting up the diarization pipeline
+- Processing audio files
+- Performing transcription and diarization
+
+### AudioProcessor
+
+The `AudioProcessor` class (in `pyscript/audio_processing.py`) manages audio file preprocessing, including:
+
+- Loading audio files
+- Resampling
+- Converting to WAV format
+- Displaying audio file details and changes
+- Audio enhancement (noise reduction, voice enhancement, volume boost)
+
+### AudioRecording
+
+The `audio_recording.py` module provides functions for recording audio from a microphone, checking input devices, and saving audio files.
+
+## Contributing
+
+Contributions are welcome! Please follow these steps:
+
+1. Fork the repository
+2. Create a new branch: `git checkout -b feature-branch-name`
+3. Make your changes and commit them: `git commit -m 'Add some feature'`
+4. Push to the branch: `git push origin feature-branch-name`
+5. Submit a pull request
+
+## Acknowledgments
+
+- OpenAI for the Whisper model
+- PyAnnote for the speaker diarization pipeline
+- All contributors and users of this project

demo.py

@@ -0,0 +1,47 @@
+import gradio as gr
+from pyscript import Transcriptor
+
+demo = gr.Blocks()
+transcriptor = Transcriptor(model_size="large-v3-turbo")
+
+microphone_transcribe = gr.Interface(
+    fn=transcriptor.transcribe_audio,
+    inputs=[
+        gr.Audio(sources="microphone", type="filepath", label="Microphone"),
+        gr.Radio([True, False], value=True, label="Enable audio enhancement"),
+    ],
+    outputs=[
+        gr.Textbox(label="Transcription"),
+        # gr.File(label="Download Transcription"),
+        # gr.Textbox(label="Console Output", lines=10)
+    ],
+    title="Audio-Transcription leveraging Whisper Model",
+    description=(
+        "Transcribe microphone recording or audio inputs and return the transcription with speaker diarization."
+    ),
+    allow_flagging="never",
+)
+
+file_transcribe = gr.Interface(
+    fn=transcriptor.transcribe_audio,
+    inputs=[
+        gr.Audio(sources="upload", type="filepath", label="Audio file"),
+        gr.Radio([True, False], value=True, label="Enable audio enhancement"),
+    ],
+    outputs=[
+        gr.Textbox(label="Transcription"),
+        # gr.File(label="Download Transcription"),
+        # gr.Textbox(label="Console Output", lines=10)
+    ],
+    title="Audio-Transcription leveraging Whisper Model",
+    description=(
+        "Transcribe microphone recording or audio inputs and return the transcription with speaker diarization."
+    ),
+    allow_flagging="never",
+)
+
+
+with demo:
+    gr.TabbedInterface([microphone_transcribe, file_transcribe], ["Microphone", "Audio file"])
+
+demo.queue().launch()

pyscript/__init__.py

@@ -0,0 +1,3 @@
+from .transcriptor import Transcriptor
+from .audio_processing import AudioProcessor
+__all__ = ["Transcriptor", "AudioProcessor"]

pyscript/audio_processing.py

@@ -0,0 +1,193 @@
+import os
+import librosa
+import numpy as np
+from tabulate import tabulate
+import soundfile as sf
+import scipy.ndimage
+import itertools
+from scipy.stats import pearsonr
+from tqdm import tqdm
+
+class AudioProcessor:
+
+    def __init__(self, audio_file):
+         self.path = audio_file
+         self.name = os.path.splitext(os.path.basename(audio_file))[0]
+         self.format = os.path.splitext(os.path.basename(audio_file))[1]
+         self.duration = librosa.get_duration(path=audio_file)
+         self.sample_rate = librosa.get_samplerate(audio_file)
+         self.changes = []
+         self.optimized_params = None
+         self.load_details()
+
+    # File information methods
+    def load_details(self):
+        """Save the attributes of the audio file."""
+        data = [
+            ["File Name", self.name],
+            ["File Format", self.format],
+            ["Duration", f"{self.duration} seconds"],
+            ["Sample Rate", f"{self.sample_rate} Hz"]
+        ]
+        table = tabulate(data, headers=["Attribute", "Value"], tablefmt="outline")
+        self.changes.append(table)
+        return table
+    
+    def display_details(self):
+        """Display the details of the audio file."""
+        print(self.changes[-1])
+
+    def display_changes(self):
+        """Display the changes made to the audio file side by side."""
+        self._clean_duplicates_changes()
+        if len(self.changes) == 1:
+            self.display_details()
+        else:
+            table1 = self.changes[0].split('\n')
+            table2 = self.changes[-1].split('\n')
+
+            combined_table = []
+            for line1, line2 in zip(table1, table2):
+                combined_table.append([line1, '===>', line2])
+
+            print(tabulate(combined_table, tablefmt="plain"))
+
+    def _clean_duplicates_changes(self):
+        """Remove duplicate consecutive changes from the audio file."""
+        self.changes = [change for i, change in enumerate(self.changes) 
+                        if i == 0 or change != self.changes[i-1]]
+
+    # Audio processing methods
+    def load_as_array(self, sample_rate: int = 16000) -> np.ndarray:
+        """
+        Load an audio file and convert it into a NumPy array.
+
+        Parameters
+        ----------
+        sample_rate : int, optional
+            The sample rate to which the audio will be resampled (default is 16000 Hz).
+
+        Returns
+        -------
+        np.ndarray
+            A NumPy array containing the audio data.
+        """
+        try:
+            audio, sr = librosa.load(self.path, sr=sample_rate)
+            self.sample_rate = sr
+            return audio
+        except Exception as e:
+            raise RuntimeError(f"Failed to load audio file: {e}")
+        
+    def resample_wav(self) -> str:
+        output_path = os.path.join('resampled_files', f'{self.name}.wav')
+        try:
+            audio, sr = librosa.load(self.path)
+            resampled_audio = librosa.resample(y=audio, orig_sr=sr, target_sr=16000)
+            os.makedirs(os.path.dirname(output_path), exist_ok=True)
+            sf.write(output_path, resampled_audio, 16000)
+            self._update_file_info(output_path)
+            return output_path
+        except Exception as e:
+            raise RuntimeError(f"Failed to resample audio file: {e}")
+        
+    def convert_to_wav(self):
+        """
+        Converts an audio file to WAV format.
+
+        Returns
+        -------
+        str
+            The path to the converted audio file.
+        """        
+        output_path = os.path.join('converted_files', f'{self.name}.wav')
+        try:
+            os.makedirs(os.path.dirname(output_path), exist_ok=True)
+            audio, sr = librosa.load(self.path, sr=16000)
+            sf.write(output_path, audio, 16000)
+            self._update_file_info(output_path)
+            return output_path
+        except Exception as e:
+            raise RuntimeError(f"Failed to convert audio file to WAV: {e}")
+
+    def enhance_audio(self, noise_reduce_strength=0.5, voice_enhance_strength=1.5, volume_boost=1.2):
+        """
+        Enhance audio quality by reducing noise and clarifying voices.
+        """
+        try:
+            y, sr = librosa.load(self.path, sr=16000)
+            y_enhanced = self._enhance_audio_sample(y, noise_reduce_strength, voice_enhance_strength, volume_boost)
+
+            output_path = os.path.join('enhanced_files', f'{self.name}_enhanced.wav')
+            os.makedirs(os.path.dirname(output_path), exist_ok=True)
+            sf.write(output_path, y_enhanced, sr)
+
+            self._update_file_info(output_path)
+            return output_path
+        except Exception as e:
+            raise RuntimeError(f"Failed to enhance audio: {e}")
+
+    def optimize_enhancement_parameters(self, step=0.25, max_iterations=50, sample_duration=30):
+        """
+        Find optimal parameters for audio enhancement using grid search on a sample.
+        """
+        y_orig, sr = librosa.load(self.path, duration=sample_duration)
+        
+        param_ranges = [
+            np.arange(0.25, 1.5, step),  # noise_reduce_strength
+            np.arange(1.0, 3.0, step),   # voice_enhance_strength
+            np.arange(1.0, 2.0, step)    # volume_boost
+        ]
+
+        best_score = float('-inf')
+        best_params = None
+        
+        total_iterations = min(max_iterations, len(list(itertools.product(*param_ranges))))
+        
+        for params in tqdm(itertools.islice(itertools.product(*param_ranges), max_iterations), 
+                           total=total_iterations, 
+                           desc="Searching for optimal parameters"):
+            y_enhanced = self._enhance_audio_sample(y_orig, *params)
+            
+            min_length = min(len(y_orig), len(y_enhanced))
+            y_orig_trimmed = y_orig[:min_length]
+            y_enhanced_trimmed = y_enhanced[:min_length]
+            
+            correlation, _ = pearsonr(y_orig_trimmed, y_enhanced_trimmed)
+            
+            S_orig = np.abs(librosa.stft(y_orig_trimmed))
+            S_enhanced = np.abs(librosa.stft(y_enhanced_trimmed))
+            contrast_improvement = np.mean(librosa.feature.spectral_contrast(S=S_enhanced)) - np.mean(librosa.feature.spectral_contrast(S=S_orig))
+            
+            score = correlation + 0.5 * contrast_improvement
+            
+            if score > best_score:
+                best_score = score
+                best_params = params
+
+        self.optimized_params = best_params
+        return best_params
+
+    def _enhance_audio_sample(self, y, noise_reduce_strength=0.5, voice_enhance_strength=1.5, volume_boost=1.2):
+        S = librosa.stft(y)
+        S_mag, S_phase = np.abs(S), np.angle(S)
+        S_filtered = scipy.ndimage.median_filter(S_mag, size=(1, 31))
+        
+        mask = np.clip((S_mag - S_filtered) / (S_mag + 1e-10), 0, 1) ** noise_reduce_strength
+        S_denoised = S_mag * mask * np.exp(1j * S_phase)
+
+        y_denoised = librosa.istft(S_denoised)
+
+        y_harmonic, y_percussive = librosa.effects.hpss(y_denoised)
+        y_enhanced = (y_harmonic * voice_enhance_strength + y_percussive) * volume_boost
+
+        return librosa.util.normalize(y_enhanced, norm=np.inf, threshold=1.0)
+
+    # Helper method
+    def _update_file_info(self, new_path):
+        """Update file information after processing."""
+        self.path = new_path
+        self.sample_rate = librosa.get_samplerate(new_path)
+        self.format = os.path.splitext(new_path)[1]
+        self.duration = librosa.get_duration(path=new_path)
+        self.load_details()

pyscript/audio_recording.py

@@ -0,0 +1,77 @@
+import speech_recognition as sr
+import os
+import datetime
+from termcolor import colored
+from tabulate import tabulate
+
+def micro_recording(save_folder_path: str = "audio_files", file_name: str = None, device_index: int = 0) -> str:
+    """Records audio from a microphone and saves it to a designated file."""
+    r = sr.Recognizer()
+    mic = sr.Microphone(device_index=device_index)
+
+    print_colored_separator("Starting microphone recording...", "green")
+    
+    with mic as source:
+        print_colored("Recording...", "yellow")
+        audio = r.listen(source)
+        print_colored("Recording finished.", "green")
+    
+    saved_path = save_audio_file(audio, save_folder_path, file_name)
+    
+    print_colored_separator(f"Audio file saved to: {saved_path}", "green")
+    return saved_path
+
+def check_input_device(test_duration: int = 1) -> dict:
+    """Checks the available microphone devices."""
+    devices = sr.Microphone.list_microphone_names()
+    available_devices, non_working_devices = [], []
+
+    for i, device in enumerate(devices):
+        try:
+            with sr.Microphone(device_index=i) as source:
+                sr.Recognizer().listen(source, timeout=test_duration)
+            available_devices.append(device)
+        except sr.WaitTimeoutError:
+            non_working_devices.append(device)
+        except Exception as e:
+            print(f"An error occurred while testing device {device}: {e}")
+
+    print_device_table("Available Devices", available_devices)
+    print_device_table("Non-Working Devices", non_working_devices)
+
+    return {'available_devices': available_devices, 'non_working_devices': non_working_devices}
+
+def save_audio_file(audio, save_folder_path: str, file_name: str = None) -> str:
+    """Saves the audio file to the specified path."""
+    os.makedirs(save_folder_path, exist_ok=True)
+    
+    if not file_name:
+        timestamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
+        file_name = f"recording_{timestamp}.wav"
+    else:
+        file_name = f"{file_name}.wav"
+    
+    saved_path = os.path.join(save_folder_path, file_name)
+    
+    with open(saved_path, "wb") as f:
+        f.write(audio.get_wav_data())
+    
+    print_colored("Saving audio file...", "yellow")
+    return saved_path
+
+def print_colored(message: str, color: str):
+    """Prints a colored message."""
+    print(colored(message, color))
+
+def print_colored_separator(message: str, color: str):
+    """Prints a colored message with separators."""
+    print("--------------------------------")
+    print_colored(message, color)
+    print("--------------------------------")
+
+def print_device_table(title: str, devices: list):
+    """Prints a table of devices."""
+    device_table = [[i+1, device] for i, device in enumerate(devices)]
+    print(f"\n{title}:")
+    print(tabulate(device_table, headers=["Index", "Device Name"]))
+

pyscript/transcription.py

@@ -0,0 +1,110 @@
+import os
+from itertools import cycle
+from termcolor import colored
+
+class Transcription:
+    """
+    A class for storing and saving transcriptions.
+
+    Attributes:
+    -----------
+    audio_file_path : str
+        The path to the audio file that was transcribed.
+    filename : str
+        The name of the audio file, without the extension.
+    transcriptions : list[str]
+        A list of tuples containing the speaker's label and their corresponding transcription, grouped by speaker.
+    speaker_names : dict
+        A dictionary mapping speaker labels to their assigned names.
+    segments : list
+        A list of segments from diarization.
+
+    """
+
+    def __init__(self, audio_file_path: str, transcriptions: list[str], segments: list[str]):
+        self.audio_file_path = audio_file_path
+        self.filename = os.path.splitext(os.path.basename(audio_file_path))[0]
+        self.transcriptions = self.group_by_speaker(transcriptions)
+        self.speaker_names = {}
+        self.segments = segments
+        self.colors = cycle(['red', 'green', 'blue', 'magenta', 'cyan', 'yellow'])
+
+    def __repr__(self) -> str:
+        result = []
+        for speaker, text in self.transcriptions:
+            speaker_name = self.speaker_names.get(speaker, speaker)
+            result.append(f"{speaker_name}:\n{text}")
+        return "\n\n".join(result)
+
+    def group_by_speaker(self, transcriptions: list[str]) -> list[str]:
+        """
+        Groups transcriptions by speaker.
+
+        Parameters
+        ----------
+        transcriptions : list[str]
+            A list of tuples containing the speaker's label and their corresponding transcription.
+
+        Returns
+        -------
+        list[str]
+            A list of tuples containing the speaker's label and their corresponding transcription, grouped by speaker.
+        """
+        speaker_transcriptions = []
+        previous_speaker = transcriptions[0][0]
+        speaker_text = ""
+        for speaker, text in transcriptions:
+            if speaker == previous_speaker:
+                speaker_text += text
+            else:
+                speaker_transcriptions.append((previous_speaker, speaker_text))
+                speaker_text = text
+                previous_speaker = speaker
+        speaker_transcriptions.append((previous_speaker, speaker_text))
+        return speaker_transcriptions
+
+    def save(self, directory: str = "transcripts") -> None:
+        """
+        Saves the transcription to a text file.
+
+        Parameters
+        ----------
+        directory : str, optional
+            The directory to save the transcription to. Defaults to "transcripts".
+        """
+        if not self.transcriptions:
+            raise ValueError("No transcriptions available to save.")
+        
+        os.makedirs(directory, exist_ok=True)
+        saving_path = os.path.join(directory, f"{self.filename}_transcript.txt")
+        
+        with open(saving_path, 'w', encoding='utf-8') as f:
+            for speaker, text in self.transcriptions:
+                if text:
+                    speaker_name = self.speaker_names.get(speaker, speaker)
+                    f.write(f"{speaker_name}: {text}\n")
+        
+        print(f"Transcription saved to {saving_path}")
+
+    def get_name_speakers(self) -> None:
+        """
+        Interactively assigns names to speakers in the transcriptions and retrieves the name of the speaker.
+        Provides a preview of one sentence for each speaker to help recognize who is speaking.
+        """
+        for speaker, full_text in self.transcriptions:
+            if speaker in self.speaker_names:
+                continue
+            
+            preview = full_text.split('.')[0] + '.'
+            print(f"\nCurrent speaker: {speaker}")
+            print(f"Preview: {preview}")
+            
+            new_name = input(f"Enter a name for {speaker} (or press Enter to skip): ").strip()
+            if new_name:
+                self.speaker_names[speaker] = new_name
+                print(f"Speaker {speaker} renamed to {new_name}")
+            else:
+                print(f"Skipped renaming {speaker}")
+        
+        print("\nSpeaker naming completed.")
+        print(f"Updated speaker names: {self.speaker_names}")

pyscript/transcriptor.py

@@ -0,0 +1,163 @@
+import os
+from dotenv import load_dotenv
+import whisper
+from pyannote.audio import Pipeline
+import torch
+from tqdm import tqdm
+from time import time
+from transformers import pipeline
+from .transcription import Transcription
+from .audio_processing import AudioProcessor
+
+load_dotenv()
+
+class Transcriptor:
+    """
+    A class for transcribing and diarizing audio files.
+
+    This class uses the Whisper model for transcription and the PyAnnote speaker diarization pipeline for speaker identification.
+
+    Attributes
+    ----------
+    model_size : str
+        The size of the Whisper model to use for transcription. Available options are:
+        - 'tiny': Fastest, lowest accuracy
+        - 'base': Fast, good accuracy for many use cases
+        - 'small': Balanced speed and accuracy
+        - 'medium': High accuracy, slower than smaller models
+        - 'large': High accuracy, slower and more resource-intensive
+        - 'large-v1': Improved version of the large model
+        - 'large-v2': Further improved version of the large model
+        - 'large-v3': Latest and most accurate version of the large model
+        - 'large-v3-turbo': Optimized version of the large-v3 model for faster processing
+    model : whisper.model.Whisper
+        The Whisper model for transcription.
+    pipeline : pyannote.audio.pipelines.SpeakerDiarization
+        The PyAnnote speaker diarization pipeline.
+
+    Usage:
+        >>> transcript = Transcriptor(model_size="large-v3")
+        >>> transcription = transcript.transcribe_audio("/path/to/audio.wav")
+        >>> transcription.get_name_speakers()
+        >>> transcription.save("/path/to/transcripts")
+
+    Note:
+        Larger models, especially 'large-v3', provide higher accuracy but require more 
+        computational resources and may be slower to process audio.
+    """
+
+    def __init__(self, model_size: str = "base"):
+        self.model_size = model_size
+        self.HF_TOKEN = os.getenv("HF_TOKEN")
+        if not self.HF_TOKEN:
+            raise ValueError("HF_TOKEN not found. Please store token in .env")
+        self._setup()
+
+    def _setup(self):
+        """Initialize the Whisper model and diarization pipeline."""
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        print("Initializing Whisper model...")
+        if self.model_size == "large-v3-turbo":
+            self.model = pipeline(
+                task="automatic-speech-recognition",
+                model="ylacombe/whisper-large-v3-turbo",
+                chunk_length_s=30,
+                device=self.device,
+            )
+        else:
+            self.model = whisper.load_model(self.model_size, device=self.device)
+        print("Building diarization pipeline...")
+        self.pipeline = Pipeline.from_pretrained(
+            "pyannote/speaker-diarization-3.1", 
+            use_auth_token=self.HF_TOKEN
+        ).to(torch.device(self.device))
+        print("Setup completed successfully!")
+
+    def transcribe_audio(self, audio_file_path: str, enhanced: bool = False) -> Transcription:
+        """
+        Transcribe an audio file.
+
+        Parameters:
+        -----------
+        audio_file_path : str
+            Path to the audio file to be transcribed.
+        enhanced : bool, optional
+            If True, applies audio enhancement techniques to improve transcription quality.
+            This includes noise reduction, voice enhancement, and volume boosting.
+
+        Returns:
+        --------
+        Transcription
+            A Transcription object containing the transcribed text and speaker segments.
+        """
+        try:
+            print("Processing audio file...")
+            processed_audio = self.process_audio(audio_file_path, enhanced)
+            audio_file_path = processed_audio.path
+            audio, sr, duration = processed_audio.load_as_array(), processed_audio.sample_rate, processed_audio.duration
+            
+            print("Diarization in progress...")
+            start_time = time()
+            diarization = self.perform_diarization(audio_file_path)
+            print(f"Diarization completed in {time() - start_time:.2f} seconds.")
+            segments = list(diarization.itertracks(yield_label=True))
+
+            transcriptions = self.transcribe_segments(audio, sr, duration, segments)
+            return Transcription(audio_file_path, transcriptions, segments)
+        except Exception as e:
+            raise RuntimeError(f"Failed to process the audio file: {e}")
+
+    def process_audio(self, audio_file_path: str, enhanced: bool = False) -> AudioProcessor:
+        """
+        Process the audio file to ensure it meets the requirements for transcription.
+
+        Parameters:
+        -----------
+        audio_file_path : str
+            Path to the audio file to be processed.
+        enhanced : bool, optional
+            If True, applies audio enhancement techniques to improve audio quality.
+            This includes optimizing noise reduction, voice enhancement, and volume boosting
+            parameters based on the audio characteristics.
+
+        Returns:
+        --------
+        AudioProcessor
+            An AudioProcessor object containing the processed audio file.
+        """
+        processed_audio = AudioProcessor(audio_file_path)
+        if processed_audio.format != ".wav":
+            processed_audio.convert_to_wav()
+        
+        if processed_audio.sample_rate != 16000:
+            processed_audio.resample_wav()
+        
+        if enhanced:
+            parameters = processed_audio.optimize_enhancement_parameters()
+            processed_audio.enhance_audio(noise_reduce_strength=parameters[0], 
+                                        voice_enhance_strength=parameters[1], 
+                                        volume_boost=parameters[2])
+        
+        processed_audio.display_changes()
+        return processed_audio
+
+    def perform_diarization(self, audio_file_path: str):
+        """Perform speaker diarization on the audio file."""
+        with torch.no_grad():
+            return self.pipeline(audio_file_path)
+
+    def transcribe_segments(self, audio, sr, duration, segments):
+        """Transcribe audio segments based on diarization."""
+        transcriptions = []
+        
+        for turn, _, speaker in tqdm(segments, desc="Transcribing segments", unit="segment", ncols=100, colour="green"):
+            start = turn.start
+            end = min(turn.end, duration)
+            segment = audio[int(start * sr):int(end * sr)]
+            if self.model_size == "large-v3-turbo":
+                result = self.model(segment)
+            else:
+                result = self.model.transcribe(segment, fp16=self.device == "cuda")
+            transcriptions.append((speaker, result['text'].strip()))
+        
+        return transcriptions

requirements.txt

@@ -0,0 +1,15 @@
+openai-whisper
+pyannote.audio
+librosa
+tqdm
+python-dotenv
+termcolor
+pydub
+SpeechRecognition
+pyaudio
+tabulate
+soundfile
+torch
+numpy
+transformers
+gradio