Using color information can considerably improve capabilities of image classification algorithms compared to purely intensity based approaches. However, we need a good color space in order to use color information in the same way as humans perceive color differences. We use the CIE-LUV color space here because it separates luminance and chroma information and it is easy to compute color differences in this metric. Our model is based on a classical MRF classificationmodel: The classes are represented by a Gaussian distribution and the prior distribution of the classes is modeled by a first order Markov Random Field (MRF).
The visual motion derived from a sequence of time-varying images is also a valuable source of information. Basically, it can be used to detect motion in the scene but it is also possible to derive more detailed information such as position, orientation of a visible surface or 3D reconstruction of a scene. Herein, we are interested in computing displacement vectors.
When we have a sequence of color images, still image models can be easily extended to take into account the information in the previous and next frames. Instead of a 2D neighborhood system, we can use a 3D one with inter-frame cliques. If the camera or the objects in the scene are not moving then this model yields good segmentations. In the case of moving objects, however, this static model can fail.
To overcome the problem caused by moving objects, we introduce a displacement field (DF) in order to take into account motion information in the label field. For simplicity, the DF is defined over the same lattice as the label field by placing a new lattice between two neighboring frames. DF is a vector-valued MRF giving the displacement vector at each site between two frames in the sequence. The estimation of the DF is done in parallel with the label field and no external algorithm or initialization is needed. The energy function of the so-defined system is minimized by the Metropolis algorithm which results is the classification of the input frames and the displacement vectors between frames.
Usually, MRF based classification methods suffer from a lack of parameter estimation. The majority of the proposed methods are supervised which limits their practical use because human intervention is needed to compute the model parameters.
Herein, we are interested in data driven algorithms since in real life applications, model parameters are usually unknown, one has to estimate them without human intervention. Some algorithms used nowadays are iterative, subsequently generating a labeling, estimating parameters from it, then generating a new labeling using these parameters, etc ...For such a method, we need a reasonably good initial value for each parameter. Since the classes are represented by a Gaussian distribution, the initialization of the mean vector and the covariance matrix of each class is very important because of its influence on subsequent labelings and hence on the final estimates. On the other hand, it is a classical problem, namely the determination of the components of a Gaussian mixture without any a priori information. There are many approaches in this domain: Method of moments, Prony's Method or scale space histogram analysis, for instance. The main problem with these methods is that they relay only on the histogram of the observed image. For noisy images where the histogram usually does not have clearly distinguishable peaks, these approaches are unreliable. On the other hand, the direct use of color image histograms is impossible since they are too sparse. The number of possible colors (256*256*256=16777216 considering 24 bit color codes) is usually much greater than the number of pixels yielding a flat histogram.
We propose a method to find the components of such a histogram which takes into account spatial information. The basic idea is to re-quantize the observed image via a pre-segmentation. Using this algorithm, we develop an unsupervised, motion compensated color image classification algorithm. The only parameter supplied by the user is the number of classes.