Chosen development environment: Matlab

Step by step plan:

1. Take pictures (to create a dataset) of
• actual stairs (indoor/outdoor different illuminations)
• objects that look like stair
• actual doors (indoor/outdoor)
• objects that are similar to doors
2. Approaches to investigate
• edge detection for finding the horizontal edges of stairs to identify then based on the distance between edges and their reqularity.
Enhancing the quality of edges:
• grayscale conversion
• histogram equalization
• different kernels were investigated: Sobel, Prewitt, Roberts, Canny
Questions: what is the minimal number of horizontal lines to consider the object as stair -> initially Roberts looked the most promising
• use Fourier-transform to get lines perpendiculor to the perspective parallel lines of stairs -> still a plan
3. Find/Write code for the most promising approaches
4. Test the solutions and improve then if possible. Also measure true and false positives and negatives, basically provide statistics about the success/failure ratio.

Decided tasks for each team member:

• Melinda - documentation & web page design
• Dávid - take various pictures of "street objects"
• Severino - study different possible approaches to solve the task
• Tamás - coding

General team tasks:

• daily reports of progress
• theoretical discussions in the beginning of each session
• assign tasks for the next day
• evaluate the results

Started coding different methods for stairs recognition in Matlab, such as:

• edge detection:
• instead of taking one result of an individual edge detector, we combined (added) the results of several ones
• the Canny edge detector was rejected because of the noise in the images we took
• cropping the most promising region of the input image where the stairs can be located usually if walking

• investigated the Fourier-transform of images that contained stairs and images that did not contain stairs -> we found that there was a significant difference: a rather dense concentration in the Fourier spectrum formed along a line perpendicular to the parallel edges of the staircase

Plan: take the absolute logarithm of the Fourier image and search for an almost vertical line with maximum intensity sum along the line. the code has been created in Matlab. early had shown that this could work, based on these tests an empiricaL threshold for the sum of intensity values along the line could be set

• Discussion about the results considering the stairway detection -> decided to sharpen the images a little bit to enhance edgesm otherwise results were promising even with objects that looked like stair
• The focus shifted to recognizing doors in pictures. Some possible approaches were on discussion:
• detecting corners and/or edges to identify rectangle-like shapes
• taking various pictures of different handles of doors (several angles, different illuminations, dozen of types) to use them for template matching. -> maybe extracting edge first by Hough-transform or using USURF to obtain handle descriptors to enable matching or any other pattern matching algorithm.
• All found online implementation were tested. Promising methods did exist, but due to lack of remaining time nutil the deadline we decided not to develop our own algorithm of this type. Instead we took our initial approach to investigate the Fourier transform of images containing door-like objects. The idea was to detect a cross-like pattern in the Fourier space, since our tests showed that there is a high probability that the aforementioned pattern exists if a door is present in the image. Implementation will follow the next day.

• Started to write detailed documentation.
• We organized the available pictures in groups of separate folders with different types in each folder.
• Decided not to use video sequence because eventually it would have been processed frame-by-frame which is identical to analyzing still images (possibly taken with mobile phones)
• Development of a door detection algorithm has been finished based on the Fourier spectrum of the taken image. All used images for testing have been resized to a width of ~484 and a height of approximately ~322 to exactly set the optimal thresholds in the algorithm. The applied method is similar to the one used for the stairs: a closely vertical line with the maximum absolute intensitives is located first. Of the found line has a negative slope then the search for a somewhat horizontal line with a maximal sum of intensives along that line is limited to a positive slope, and vica versa. We also restricted the search space of possible lines to consider with a certain offset, which is usually about 20-30 pixels vertically and horizontally.
The treshold values are also the same for the sum of intensitives along the almost horizontal and vertical lines detected. Since these are closely perpedicular to each other they indicate that numerous perpendiculor lines do exist in the original image as well. Based on the photos we have taken of doors and door-shaped objects we can say that this applies for almost all of them. The strongness of these lines is required to be above a defined treshold to safely assume that the image the subject took in the street/in the building etc. actually shows a real door or a door-like object. This threshold depends heavily on the size of the image, in our case 2000.0 seemed like a reasonable value. It assured a rather good performance, even when trying to distinguish between doors and objects that look like doors.

• Finalized the developed code, added necessary comments
• Applied finishing touches to the presentaion
• Finished the documentation on the web page
• Run some more tests, so we could present some more results.