Shadow Detection and Removal

Project

The presence of shadows has been responsible for reducing the reliability of many computer vision algorithms, including segmentation, object detection, scene analysis, stereo, tracking, etc. Therefore, shadow detection and removal is an important pre-processing for improving performance of such vision tasks.
Decomposition of a single image into a shadow image and a shadow-free image is a difficult problem, due to complex interactions of geometry, albedo, and illumination. Many techniques have been proposed over the years, but shadow detection still remains an extremely challenging problem, particularly from a single image.
Our goal is to create a system that can automatically detect shadow regions and then remove from simple images. While detecting all shadows is expected to remain hard, we make some constraints related to the structure of the images.
We present an algorithm to automatically detect shadows and remove them from simple consumer images. Our algorithms are developed in Matlab and C++

Shadow Detection

For shadow detection we present an approach based on statistics of intensity. We represent the picture in the YCbCr color space and we compute the histogram dissension, which gives us a more contrast image at the Y channel. After computing the full average at the Y channel, we perform a sliding window operation, to decide which pixels belongs to the shadow. The noise is then reduced by applying a median filter.
The algorithm finds the lower intensities areas in the picture, and this can describe darker objects too (false detection). For such situations, we can further decide if we have a shadow by taking into consideration that its border is smooth (the border is usually rough for objects).
The result of the shadow detection is a binary shadow mask, which will be the input to the shadow removal algorithm.
Remark: As a further improvement, we can introduce a segmentation algorithm and we can use our method for each of the segments.

Shadow Removal

Model-based shadow removal
We use a simple shadow model, where there are two types of light sources: direct and ambient light. Direct light comes directly from the source, while environment light is from reflections of surrounding surfaces. For shadow areas part or all of the direct light is occluded.
Additive shadow removal
Another,rather simple shadow removal technique was an additive correction of the color intensities in the shadow area. We computed the average pixel intensities in the shadow and lit areas of the image and added this difference to the pixels in the shadow areas.
Combined shadow removal
The third shadow removal method vas the combination of the previous two ones. We converted the images to the YCbCr color-space. After, we used the additive method for the correction on the Y channel, and the model-based method for the correction of the Cb and Cr channels.