readme.md

ASL Recognition App

This application recognizes American Sign Language (ASL) hand signs using image processing and machine learning techniques.

How it works

1. Image Upload: Users can upload an image of an ASL hand sign.
2. Hand Detection: The app uses MediaPipe to detect hand landmarks in the image.
3. Feature Extraction: Angles between hand landmarks are calculated and normalized.
4. Prediction: A Random Forest model predicts the ASL sign based on the extracted features.
5. Visualization: The app displays the detected hand landmarks and top predictions.

Supported Alphabets

The app currently works for the following ASL alphabets: A, B, C, E, F, G, H, I, J, K, L, O, Q, R, S, W, Y

The app does not support or may not work correctly for: D, M, N, P, T, U, V, X, Z

Note: The model's performance may vary and is subject to improvement.

Image Preprocessing and Feature Extraction

Image Preprocessing

• Image Loading: We use OpenCV (cv2) to load and process images.
• Color Conversion: Images are converted from BGR to RGB color space for compatibility with MediaPipe.

Hand Landmark Detection

We use MediaPipe Hands for detecting hand landmarks in the images:

• MediaPipe Hands: Initializes with static_image_mode=True, max_num_hands=1, and min_detection_confidence=0.5.
• Landmark Extraction: 21 hand landmarks are extracted from each image.

Feature Extraction

Landmark Normalization

1. Centering: Landmarks are centered by subtracting the mean position.
2. Scaling: Centered landmarks are scaled to unit variance.

Angle Calculation

We calculate angles between all pairs of landmarks:

1. Vector Calculation: For each pair of landmarks (i, j), we calculate the vector from i to j.
2. Angle Computation: We compute the arcosine of the x and y components of the normalized vector.
3. Feature Vector: The angles form a 420-dimensional feature vector (21 choose 2 = 210 pairs, 2 angles per pair).

Model Input

The preprocessed features (angles) are used as input to our Random Forest model for ASL sign classification.

This preprocessing pipeline ensures that our model receives consistent and informative features, regardless of the hand's position or size in the original image.

Usage

The "View Hand Landmarks" tab allows users to see hand landmarks for pre-loaded ASL signs. Select one or more alphabets from the supported list to view their corresponding hand landmarks.