metadata

title: BlinkBlur
emoji: 👀
sdk: gradio
sdk_version: 4.10.0
python_version: 3.9
app_file: app.py

Eye Blink / Blurriness detectors

Try BlinkBlur on HuggingFace 🤗

Packages

gradio==4.10.0
matplotlib==3.8.2
mediapipe==0.10.9
numpy==1.26.2
opencv-contrib-python==4.8.1.78
opencv-python==4.8.1.78
pandas==2.1.4
pytorch-lightning==2.1.2
scikit-learn==1.3.2
seaborn==0.13.0
torch==2.1.2
torchvision==0.16.2
tqdm==4.66.1

How to use

usage: main.py [-h] [-w] [-p PATH] [-e] [-b]

optional arguments:
  -h, --help            show this help message and exit
  -w, --webcam          Runs eye blink detection on the user's webcam.
                        Overrides other options.
  -p PATH, --path PATH  path of image to process
  -e, --eyes            Detects eye blink on the input image
  -b, --blur            Estimates image blur

You can try with images available in examples/.

Command example:

python main.py -w
python main.py --path examples/close.jpg --eyes
python main.py -p examples/open_far.jpg -e
python main.py -p examples/blurry_1.jpg --blur
python main.py -p examples/sharp_1.jpg -b

Metrics

Closed-eyes detector

Here are some evaluation metrics on the model, calculated with the code in closed_eyes_detection/training/test.py.

The confusion matrix has been calculated on a classification threshold of 0.49, this corresponds to the red cross on the curve.

My assumption is that we want to make sure we don't keep a picture where the user is blinking so we need to be more strict on it to maximize recall.

Blurriness estimator

The blurriness estimator uses the variance of the laplacian to assess the blurriness.

Here are some metrics and statistics on the blurriness detection task, calculated with the code in blurriness_estimation/calculate_metrics.py.

The red vertical line indicates the chosen threshold, given by the following metrics:

Assumptions

Input images are in RGB format, in a format readable by OpenCV/PIL
For live webcam detection, the user has at least 1 camera
The person's face is not rotated
The person's face is not too close to the sides of the frame (this can lead to a handled error during cropping)

Decisions

If several persons appear, only the first one detected will be processed
I use MediaPipe's Face Detection to get eyes coordinates, then crop the eyes region and feed them into a custom model.
The blurriness estimation is based on an edge detector (variance of laplacian). The decision is done with a threshold of 250, selected based on metrics detailed in the Metrics section.

Datasets

I trained the closed eye detection model on this kaggle dataset.
The blurriness estimator was tested on this kaggle dataset