Red Eye Correction from an Image using OpenCV .

We’ve all experienced taking photographs in which everyone ends up with bright red devil eyes. So the question arises "Why ....? "

Why Do Eyes Look Red in Photos?

In dim light or at night,  pupil very often expands due to lack of light. As camera's flash goes on, pupil doesn't have time to react (contract). Therefore, a large burst of light reaches the retina, reflects back on camera There’s a layer on the back of the eye called the choroid which is full of blood causing the reflection color to be red. 

Many cameras now have a setting, it shoots a series of flashes before the camera takes a picture allowing time for the pupil to construct accordingly. 

But the good thing is we can remove it using ImageProcessing Techniques discussed below.

How to remove Red-Eyes Automatically?

Step1: Eye detection

We will use the standard OpenCV Haar detector (haarcascade_eye.xml) for eyes detection .

To keep things simple, assume the image is a face portrait only, or image is a close-up of eyes.

Code for Eye Detector

In line 19, cascade classifier detects the eyes in the image and returns a list of the bounding rectangle of each eye region. Each element of list is of the form [x, y, w,h]

x ,y  : Top left corner position of eye Rectangle

w,h  : Width and Height of the Rectangle 

Step 2: Masking Red Eye Region

Here we need to find the part of the pupil that is affected by the red eyes.
For simplicity, we say that red channel should be greater than a threshold and also mean of the blue and green channel.

We iterate over each eye rectangle.

• Crop the eye region  
• Split the image into 3 channels (Remember OpenCV store img in BGR format by default)
• Create a new image(bg) by adding blue and green channel 
• Create Red region mask ( You can play with the threshold value )   

Masked eye

The masked region may contain some undesired region. We find the all different connected component,

• Find all contours
• Find the contours with the largest area
• Clear the mask image
• Draw contour with the largest area on mask image.
• Masked Image may contain some irregularity or holes, To fill the holes we use morphological closing operation.
• Dilate the mask region to cover a little extra region of pupil ( because the outer perimeter may not be detected ) 

Step 3: Fix Red Eyes

Now we have a perfect mask that contains the only red region of the eye. We know that the information of on;y red channel is lost.The best way to recover the information is from the other two channels (blue and green ),
A good idea is to take the mean of the blue and green channel to get the texture and replace this with all the three channels, which give a dark color pupil texture.

Step 4: Show the Result

In the previous step we fixed the eye region, to compare the result we merge both images (Original and Red Eye Corrected) horizontally to compare the results.

(a) Original Image           (b) Red Eye Corrected

It looks pretty, Isn't it ??
Now you can make your own script to correct red eye.
To download the code: Click Here (Link to Github Repository)

In this post,  I assume you are familiar enough with Python and OpenCV, so most of the code is self-explanatory. If you have any doubt or question, feel free to ask anything in the comment.
Thanks .